You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by Hartmut Lang <ha...@gmail.com> on 2014/10/23 09:41:00 UTC
JaxRs: reuse of WebTarget can cause OOM
Hi,
my question is how i can reuse (or not) the WebTarget of a JaxRs Client
with CXF 3.0.2.
My scenario is roughly like this:
Init:
Client client = ClientBuilder.newBuilder().
register(JacksonJsonProvider.class).
build();
WebTarget target = client.target(baseUrl).path("lights/{lightNum}/{state}");
Then i issue several (thousands) requests like this:
Invocation.Builder builder = target.
resolveTemplate("lightNum", lightNum).
resolveTemplate("state", state).
request();
Response response = builder.put(null);
response.close();
What i see is, that for every request a new WebClient-Object is created and
added to
ClientImpl.baseClients-HashMap.
And finally i can run into OOM-issues.
My question:
- is it ok to reuse the WebTarget for several requests?
- if i close the client and create a new one, this solves this issue. Is
this the intended live-cycle of Client/WebTarget?
Thanks,
Hartmut
Re: JaxRs: reuse of WebTarget can cause OOM
Posted by Sergey Beryozkin <sb...@gmail.com>.
This should cover it:
http://git-wip-us.apache.org/repos/asf/cxf/commit/61c7b99b
Thanks, Sergey
On 31/10/14 21:18, Sergey Beryozkin wrote:
> Hi Hartmut,
> On 31/10/14 20:08, Hartmut Lang wrote:
>> Hi Sergey,
>> thanks for looking into this.
>> I will do some checks.
>> But just on looking at the change you did let me ask this:
>> now using the weak-reference the close will only be called on some of the
>> WebClients (those which are not yet GCed), right? Does this make sense?
>>
> You are right. WebTarget delegates to a CXF specific WebClient, every
> WebClient operates on the instance of CXF HttpConduit. I think not
> having all of WebTargets being closed pro-actively is not a major problem.
> The only problem I can imagine is that when WebTarget is closed via a
> Client.close(), the underlying WebClient would also notify various
> listeners, etc...
> It might make sense to have WebTarget finalize() implemented carefully
> to ensure the underlying WebClient.close() is called...
> I'll think about it and update you when I'm done...
>
> Cheers, Sergey
>
>> Hartmut
>>
>> 2014-10-28 13:46 GMT+01:00 Sergey Beryozkin <sb...@gmail.com>:
>>
>>> Hi
>>>
>>> I updated ClientImpl to use WeakHashMap, see
>>>
>>> https://issues.apache.org/jira/browse/CXF-6066
>>> and
>>> http://git-wip-us.apache.org/repos/asf/cxf/commit/c9e85e76
>>>
>>> Can you please experiment with 3.0.3-SNAPSHOT a bit later on just to
>>> confirm the issue has gone away ?
>>>
>>> Thanks, Sergey
>>>
>>> On 28/10/14 11:22, Sergey Beryozkin wrote:
>>>
>>>> Hi
>>>>
>>>> I left right after I replied, sorry for a delay...
>>>> I'm going to deal with this issue right now
>>>>
>>>> Thanks, Sergey
>>>> On 23/10/14 11:26, Hartmut Lang wrote:
>>>>
>>>>> Hi,
>>>>> thanks for your response.
>>>>> Should i open an issue for this?
>>>>>
>>>>> Hartmut
>>>>>
>>>>> 2014-10-23 10:43 GMT+02:00 Sergey Beryozkin <sb...@gmail.com>:
>>>>>
>>>>> Hi
>>>>>> On 23/10/14 08:41, Hartmut Lang wrote:
>>>>>>
>>>>>> Hi,
>>>>>>>
>>>>>>> my question is how i can reuse (or not) the WebTarget of a JaxRs
>>>>>>> Client
>>>>>>> with CXF 3.0.2.
>>>>>>>
>>>>>>> My scenario is roughly like this:
>>>>>>>
>>>>>>> Init:
>>>>>>> Client client = ClientBuilder.newBuilder().
>>>>>>> register(JacksonJsonProvider.class).
>>>>>>> build();
>>>>>>> WebTarget target = client.target(baseUrl).path("
>>>>>>> lights/{lightNum}/{state}");
>>>>>>>
>>>>>>>
>>>>>>> Then i issue several (thousands) requests like this:
>>>>>>>
>>>>>>> Invocation.Builder builder = target.
>>>>>>> resolveTemplate("lightNum", lightNum).
>>>>>>> resolveTemplate("state", state).
>>>>>>> request();
>>>>>>> Response response = builder.put(null);
>>>>>>> response.close();
>>>>>>>
>>>>>>> What i see is, that for every request a new WebClient-Object is
>>>>>>> created
>>>>>>> and
>>>>>>> added to
>>>>>>> ClientImpl.baseClients-HashMap.
>>>>>>> And finally i can run into OOM-issues.
>>>>>>>
>>>>>>> Right, that was the quick implementation to support the
>>>>>>> Client.close()
>>>>>> docs saying:
>>>>>>
>>>>>> "Calling this method effectively invalidates all resource targets
>>>>>> produced
>>>>>> by the client instance. Invoking any method on such targets once the
>>>>>> client
>>>>>> is closed would result in an IllegalStateException being thrown."
>>>>>>
>>>>>> To be honest the fact most of WebTarget methods create new WebTargets
>>>>>> may
>>>>>> indeed lead to some unexpected results, though in this case the bug
>>>>>> is in
>>>>>> the CXF code, i.e, I guess it has to be a WeakHashMap, I'll take care
>>>>>> of it
>>>>>> next week as I'm off shortly...
>>>>>>
>>>>>>
>>>>>>> My question:
>>>>>>> - is it ok to reuse the WebTarget for several requests?
>>>>>>>
>>>>>>>
>>>>>> I guess for now it is better to reuse for a small number of requests
>>>>>>
>>>>>> - if i close the client and create a new one, this solves this
>>>>>> issue. Is
>>>>>>
>>>>>>> this the intended live-cycle of Client/WebTarget?
>>>>>>>
>>>>>>>
>>>>>> yes, this is my understanding.
>>>>>>
>>>>>> Thanks, Sergey
>>>>>>
>>>>>>
>>>>>>> Thanks,
>>>>>>> Hartmut
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> --
>>> Sergey Beryozkin
>>>
>>> Talend Community Coders
>>> http://coders.talend.com/
>>>
>>> Blog: http://sberyozkin.blogspot.com
>>>
>>
>
Re: JaxRs: reuse of WebTarget can cause OOM
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi Hartmut,
On 31/10/14 20:08, Hartmut Lang wrote:
> Hi Sergey,
> thanks for looking into this.
> I will do some checks.
> But just on looking at the change you did let me ask this:
> now using the weak-reference the close will only be called on some of the
> WebClients (those which are not yet GCed), right? Does this make sense?
>
You are right. WebTarget delegates to a CXF specific WebClient, every
WebClient operates on the instance of CXF HttpConduit. I think not
having all of WebTargets being closed pro-actively is not a major problem.
The only problem I can imagine is that when WebTarget is closed via a
Client.close(), the underlying WebClient would also notify various
listeners, etc...
It might make sense to have WebTarget finalize() implemented carefully
to ensure the underlying WebClient.close() is called...
I'll think about it and update you when I'm done...
Cheers, Sergey
> Hartmut
>
> 2014-10-28 13:46 GMT+01:00 Sergey Beryozkin <sb...@gmail.com>:
>
>> Hi
>>
>> I updated ClientImpl to use WeakHashMap, see
>>
>> https://issues.apache.org/jira/browse/CXF-6066
>> and
>> http://git-wip-us.apache.org/repos/asf/cxf/commit/c9e85e76
>>
>> Can you please experiment with 3.0.3-SNAPSHOT a bit later on just to
>> confirm the issue has gone away ?
>>
>> Thanks, Sergey
>>
>> On 28/10/14 11:22, Sergey Beryozkin wrote:
>>
>>> Hi
>>>
>>> I left right after I replied, sorry for a delay...
>>> I'm going to deal with this issue right now
>>>
>>> Thanks, Sergey
>>> On 23/10/14 11:26, Hartmut Lang wrote:
>>>
>>>> Hi,
>>>> thanks for your response.
>>>> Should i open an issue for this?
>>>>
>>>> Hartmut
>>>>
>>>> 2014-10-23 10:43 GMT+02:00 Sergey Beryozkin <sb...@gmail.com>:
>>>>
>>>> Hi
>>>>> On 23/10/14 08:41, Hartmut Lang wrote:
>>>>>
>>>>> Hi,
>>>>>>
>>>>>> my question is how i can reuse (or not) the WebTarget of a JaxRs Client
>>>>>> with CXF 3.0.2.
>>>>>>
>>>>>> My scenario is roughly like this:
>>>>>>
>>>>>> Init:
>>>>>> Client client = ClientBuilder.newBuilder().
>>>>>> register(JacksonJsonProvider.class).
>>>>>> build();
>>>>>> WebTarget target = client.target(baseUrl).path("
>>>>>> lights/{lightNum}/{state}");
>>>>>>
>>>>>>
>>>>>> Then i issue several (thousands) requests like this:
>>>>>>
>>>>>> Invocation.Builder builder = target.
>>>>>> resolveTemplate("lightNum", lightNum).
>>>>>> resolveTemplate("state", state).
>>>>>> request();
>>>>>> Response response = builder.put(null);
>>>>>> response.close();
>>>>>>
>>>>>> What i see is, that for every request a new WebClient-Object is created
>>>>>> and
>>>>>> added to
>>>>>> ClientImpl.baseClients-HashMap.
>>>>>> And finally i can run into OOM-issues.
>>>>>>
>>>>>> Right, that was the quick implementation to support the Client.close()
>>>>> docs saying:
>>>>>
>>>>> "Calling this method effectively invalidates all resource targets
>>>>> produced
>>>>> by the client instance. Invoking any method on such targets once the
>>>>> client
>>>>> is closed would result in an IllegalStateException being thrown."
>>>>>
>>>>> To be honest the fact most of WebTarget methods create new WebTargets
>>>>> may
>>>>> indeed lead to some unexpected results, though in this case the bug
>>>>> is in
>>>>> the CXF code, i.e, I guess it has to be a WeakHashMap, I'll take care
>>>>> of it
>>>>> next week as I'm off shortly...
>>>>>
>>>>>
>>>>>> My question:
>>>>>> - is it ok to reuse the WebTarget for several requests?
>>>>>>
>>>>>>
>>>>> I guess for now it is better to reuse for a small number of requests
>>>>>
>>>>> - if i close the client and create a new one, this solves this
>>>>> issue. Is
>>>>>
>>>>>> this the intended live-cycle of Client/WebTarget?
>>>>>>
>>>>>>
>>>>> yes, this is my understanding.
>>>>>
>>>>> Thanks, Sergey
>>>>>
>>>>>
>>>>>> Thanks,
>>>>>> Hartmut
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>> --
>> Sergey Beryozkin
>>
>> Talend Community Coders
>> http://coders.talend.com/
>>
>> Blog: http://sberyozkin.blogspot.com
>>
>
Re: JaxRs: reuse of WebTarget can cause OOM
Posted by Hartmut Lang <ha...@gmail.com>.
Hi Sergey,
thanks for looking into this.
I will do some checks.
But just on looking at the change you did let me ask this:
now using the weak-reference the close will only be called on some of the
WebClients (those which are not yet GCed), right? Does this make sense?
Hartmut
2014-10-28 13:46 GMT+01:00 Sergey Beryozkin <sb...@gmail.com>:
> Hi
>
> I updated ClientImpl to use WeakHashMap, see
>
> https://issues.apache.org/jira/browse/CXF-6066
> and
> http://git-wip-us.apache.org/repos/asf/cxf/commit/c9e85e76
>
> Can you please experiment with 3.0.3-SNAPSHOT a bit later on just to
> confirm the issue has gone away ?
>
> Thanks, Sergey
>
> On 28/10/14 11:22, Sergey Beryozkin wrote:
>
>> Hi
>>
>> I left right after I replied, sorry for a delay...
>> I'm going to deal with this issue right now
>>
>> Thanks, Sergey
>> On 23/10/14 11:26, Hartmut Lang wrote:
>>
>>> Hi,
>>> thanks for your response.
>>> Should i open an issue for this?
>>>
>>> Hartmut
>>>
>>> 2014-10-23 10:43 GMT+02:00 Sergey Beryozkin <sb...@gmail.com>:
>>>
>>> Hi
>>>> On 23/10/14 08:41, Hartmut Lang wrote:
>>>>
>>>> Hi,
>>>>>
>>>>> my question is how i can reuse (or not) the WebTarget of a JaxRs Client
>>>>> with CXF 3.0.2.
>>>>>
>>>>> My scenario is roughly like this:
>>>>>
>>>>> Init:
>>>>> Client client = ClientBuilder.newBuilder().
>>>>> register(JacksonJsonProvider.class).
>>>>> build();
>>>>> WebTarget target = client.target(baseUrl).path("
>>>>> lights/{lightNum}/{state}");
>>>>>
>>>>>
>>>>> Then i issue several (thousands) requests like this:
>>>>>
>>>>> Invocation.Builder builder = target.
>>>>> resolveTemplate("lightNum", lightNum).
>>>>> resolveTemplate("state", state).
>>>>> request();
>>>>> Response response = builder.put(null);
>>>>> response.close();
>>>>>
>>>>> What i see is, that for every request a new WebClient-Object is created
>>>>> and
>>>>> added to
>>>>> ClientImpl.baseClients-HashMap.
>>>>> And finally i can run into OOM-issues.
>>>>>
>>>>> Right, that was the quick implementation to support the Client.close()
>>>> docs saying:
>>>>
>>>> "Calling this method effectively invalidates all resource targets
>>>> produced
>>>> by the client instance. Invoking any method on such targets once the
>>>> client
>>>> is closed would result in an IllegalStateException being thrown."
>>>>
>>>> To be honest the fact most of WebTarget methods create new WebTargets
>>>> may
>>>> indeed lead to some unexpected results, though in this case the bug
>>>> is in
>>>> the CXF code, i.e, I guess it has to be a WeakHashMap, I'll take care
>>>> of it
>>>> next week as I'm off shortly...
>>>>
>>>>
>>>>> My question:
>>>>> - is it ok to reuse the WebTarget for several requests?
>>>>>
>>>>>
>>>> I guess for now it is better to reuse for a small number of requests
>>>>
>>>> - if i close the client and create a new one, this solves this
>>>> issue. Is
>>>>
>>>>> this the intended live-cycle of Client/WebTarget?
>>>>>
>>>>>
>>>> yes, this is my understanding.
>>>>
>>>> Thanks, Sergey
>>>>
>>>>
>>>>> Thanks,
>>>>> Hartmut
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
> --
> Sergey Beryozkin
>
> Talend Community Coders
> http://coders.talend.com/
>
> Blog: http://sberyozkin.blogspot.com
>
Re: JaxRs: reuse of WebTarget can cause OOM
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi
I updated ClientImpl to use WeakHashMap, see
https://issues.apache.org/jira/browse/CXF-6066
and
http://git-wip-us.apache.org/repos/asf/cxf/commit/c9e85e76
Can you please experiment with 3.0.3-SNAPSHOT a bit later on just to
confirm the issue has gone away ?
Thanks, Sergey
On 28/10/14 11:22, Sergey Beryozkin wrote:
> Hi
>
> I left right after I replied, sorry for a delay...
> I'm going to deal with this issue right now
>
> Thanks, Sergey
> On 23/10/14 11:26, Hartmut Lang wrote:
>> Hi,
>> thanks for your response.
>> Should i open an issue for this?
>>
>> Hartmut
>>
>> 2014-10-23 10:43 GMT+02:00 Sergey Beryozkin <sb...@gmail.com>:
>>
>>> Hi
>>> On 23/10/14 08:41, Hartmut Lang wrote:
>>>
>>>> Hi,
>>>>
>>>> my question is how i can reuse (or not) the WebTarget of a JaxRs Client
>>>> with CXF 3.0.2.
>>>>
>>>> My scenario is roughly like this:
>>>>
>>>> Init:
>>>> Client client = ClientBuilder.newBuilder().
>>>> register(JacksonJsonProvider.class).
>>>> build();
>>>> WebTarget target = client.target(baseUrl).path("
>>>> lights/{lightNum}/{state}");
>>>>
>>>>
>>>> Then i issue several (thousands) requests like this:
>>>>
>>>> Invocation.Builder builder = target.
>>>> resolveTemplate("lightNum", lightNum).
>>>> resolveTemplate("state", state).
>>>> request();
>>>> Response response = builder.put(null);
>>>> response.close();
>>>>
>>>> What i see is, that for every request a new WebClient-Object is created
>>>> and
>>>> added to
>>>> ClientImpl.baseClients-HashMap.
>>>> And finally i can run into OOM-issues.
>>>>
>>> Right, that was the quick implementation to support the Client.close()
>>> docs saying:
>>>
>>> "Calling this method effectively invalidates all resource targets
>>> produced
>>> by the client instance. Invoking any method on such targets once the
>>> client
>>> is closed would result in an IllegalStateException being thrown."
>>>
>>> To be honest the fact most of WebTarget methods create new WebTargets
>>> may
>>> indeed lead to some unexpected results, though in this case the bug
>>> is in
>>> the CXF code, i.e, I guess it has to be a WeakHashMap, I'll take care
>>> of it
>>> next week as I'm off shortly...
>>>
>>>>
>>>> My question:
>>>> - is it ok to reuse the WebTarget for several requests?
>>>>
>>>
>>> I guess for now it is better to reuse for a small number of requests
>>>
>>> - if i close the client and create a new one, this solves this
>>> issue. Is
>>>> this the intended live-cycle of Client/WebTarget?
>>>>
>>>
>>> yes, this is my understanding.
>>>
>>> Thanks, Sergey
>>>
>>>>
>>>> Thanks,
>>>> Hartmut
>>>>
>>>>
>>>
>>
>
--
Sergey Beryozkin
Talend Community Coders
http://coders.talend.com/
Blog: http://sberyozkin.blogspot.com
Re: JaxRs: reuse of WebTarget can cause OOM
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi
I left right after I replied, sorry for a delay...
I'm going to deal with this issue right now
Thanks, Sergey
On 23/10/14 11:26, Hartmut Lang wrote:
> Hi,
> thanks for your response.
> Should i open an issue for this?
>
> Hartmut
>
> 2014-10-23 10:43 GMT+02:00 Sergey Beryozkin <sb...@gmail.com>:
>
>> Hi
>> On 23/10/14 08:41, Hartmut Lang wrote:
>>
>>> Hi,
>>>
>>> my question is how i can reuse (or not) the WebTarget of a JaxRs Client
>>> with CXF 3.0.2.
>>>
>>> My scenario is roughly like this:
>>>
>>> Init:
>>> Client client = ClientBuilder.newBuilder().
>>> register(JacksonJsonProvider.class).
>>> build();
>>> WebTarget target = client.target(baseUrl).path("
>>> lights/{lightNum}/{state}");
>>>
>>>
>>> Then i issue several (thousands) requests like this:
>>>
>>> Invocation.Builder builder = target.
>>> resolveTemplate("lightNum", lightNum).
>>> resolveTemplate("state", state).
>>> request();
>>> Response response = builder.put(null);
>>> response.close();
>>>
>>> What i see is, that for every request a new WebClient-Object is created
>>> and
>>> added to
>>> ClientImpl.baseClients-HashMap.
>>> And finally i can run into OOM-issues.
>>>
>> Right, that was the quick implementation to support the Client.close()
>> docs saying:
>>
>> "Calling this method effectively invalidates all resource targets produced
>> by the client instance. Invoking any method on such targets once the client
>> is closed would result in an IllegalStateException being thrown."
>>
>> To be honest the fact most of WebTarget methods create new WebTargets may
>> indeed lead to some unexpected results, though in this case the bug is in
>> the CXF code, i.e, I guess it has to be a WeakHashMap, I'll take care of it
>> next week as I'm off shortly...
>>
>>>
>>> My question:
>>> - is it ok to reuse the WebTarget for several requests?
>>>
>>
>> I guess for now it is better to reuse for a small number of requests
>>
>> - if i close the client and create a new one, this solves this issue. Is
>>> this the intended live-cycle of Client/WebTarget?
>>>
>>
>> yes, this is my understanding.
>>
>> Thanks, Sergey
>>
>>>
>>> Thanks,
>>> Hartmut
>>>
>>>
>>
>
Re: JaxRs: reuse of WebTarget can cause OOM
Posted by Hartmut Lang <ha...@gmail.com>.
Hi,
thanks for your response.
Should i open an issue for this?
Hartmut
2014-10-23 10:43 GMT+02:00 Sergey Beryozkin <sb...@gmail.com>:
> Hi
> On 23/10/14 08:41, Hartmut Lang wrote:
>
>> Hi,
>>
>> my question is how i can reuse (or not) the WebTarget of a JaxRs Client
>> with CXF 3.0.2.
>>
>> My scenario is roughly like this:
>>
>> Init:
>> Client client = ClientBuilder.newBuilder().
>> register(JacksonJsonProvider.class).
>> build();
>> WebTarget target = client.target(baseUrl).path("
>> lights/{lightNum}/{state}");
>>
>>
>> Then i issue several (thousands) requests like this:
>>
>> Invocation.Builder builder = target.
>> resolveTemplate("lightNum", lightNum).
>> resolveTemplate("state", state).
>> request();
>> Response response = builder.put(null);
>> response.close();
>>
>> What i see is, that for every request a new WebClient-Object is created
>> and
>> added to
>> ClientImpl.baseClients-HashMap.
>> And finally i can run into OOM-issues.
>>
> Right, that was the quick implementation to support the Client.close()
> docs saying:
>
> "Calling this method effectively invalidates all resource targets produced
> by the client instance. Invoking any method on such targets once the client
> is closed would result in an IllegalStateException being thrown."
>
> To be honest the fact most of WebTarget methods create new WebTargets may
> indeed lead to some unexpected results, though in this case the bug is in
> the CXF code, i.e, I guess it has to be a WeakHashMap, I'll take care of it
> next week as I'm off shortly...
>
>>
>> My question:
>> - is it ok to reuse the WebTarget for several requests?
>>
>
> I guess for now it is better to reuse for a small number of requests
>
> - if i close the client and create a new one, this solves this issue. Is
>> this the intended live-cycle of Client/WebTarget?
>>
>
> yes, this is my understanding.
>
> Thanks, Sergey
>
>>
>> Thanks,
>> Hartmut
>>
>>
>
Re: JaxRs: reuse of WebTarget can cause OOM
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi
On 23/10/14 08:41, Hartmut Lang wrote:
> Hi,
>
> my question is how i can reuse (or not) the WebTarget of a JaxRs Client
> with CXF 3.0.2.
>
> My scenario is roughly like this:
>
> Init:
> Client client = ClientBuilder.newBuilder().
> register(JacksonJsonProvider.class).
> build();
> WebTarget target = client.target(baseUrl).path("lights/{lightNum}/{state}");
>
>
> Then i issue several (thousands) requests like this:
>
> Invocation.Builder builder = target.
> resolveTemplate("lightNum", lightNum).
> resolveTemplate("state", state).
> request();
> Response response = builder.put(null);
> response.close();
>
> What i see is, that for every request a new WebClient-Object is created and
> added to
> ClientImpl.baseClients-HashMap.
> And finally i can run into OOM-issues.
Right, that was the quick implementation to support the Client.close()
docs saying:
"Calling this method effectively invalidates all resource targets
produced by the client instance. Invoking any method on such targets
once the client is closed would result in an IllegalStateException being
thrown."
To be honest the fact most of WebTarget methods create new WebTargets
may indeed lead to some unexpected results, though in this case the bug
is in the CXF code, i.e, I guess it has to be a WeakHashMap, I'll take
care of it next week as I'm off shortly...
>
> My question:
> - is it ok to reuse the WebTarget for several requests?
I guess for now it is better to reuse for a small number of requests
> - if i close the client and create a new one, this solves this issue. Is
> this the intended live-cycle of Client/WebTarget?
yes, this is my understanding.
Thanks, Sergey
>
> Thanks,
> Hartmut
>