You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by akabhishek1 <ma...@gmail.com> on 2020/02/07 12:16:01 UTC
Out Of Memory error - Qpid not releasing memory in failed to
acknowledge scenario
Hi Team,
We were doing testing of Qpid JMS client for failed scenario. We are
receiving message as client acknowledgment mode.
Scenario - Receive message but do not acknowledge the message.
Issue - Memory is not getting released by Qpid on failed Scenario
We could have this scenario in prodcution. We received the message, but
failed to process that message, so there will no aknowledgment. As we are
acknowledging only one success scenario.
Please find attached sample TestQpidRcvOnly.java, Total_Memory.PNG,
Sampler_Per_Thread_Allocation.PNG, Out_Of_Memory_Error.log .
Steps to reproduce this issue -->
1. Provide Correct details of QUEUE_NAME, SBUS_NAME, USERNAME, USERNAME in
TestQpidRcvOnly class
2. Publish 200000 message on QUEUE_NAME, from any utility
3. Start the TestQpidRcvOnly class and keep it running
4. Open JVisualVM and have a look on Heap memory
5. Wait for 20m, you can see memory consumption continously increasing
6. You can see Out Of Memory Error on exhaustion of defined Max heap Memory.
Could be 1hr depends on Max Size
7. You can have a look on Sampler--> Memory -->Per Thread Allocation, you
can observe that Qpid thread is consuming more that 98% memory.
Consumption is OK but it is not releasing the memory, which is the cause for
Out of Memory Error.
Using -- Java 8, Azure ServiceBus, Qpid-jms-client-0.48
Could you please take a look on this issue.
Regards,
Abhishek Kumar
Out_Of_Memory_Error.log
<http://qpid.2158936.n2.nabble.com/file/t396358/Out_Of_Memory_Error.log>
Sampler_Per_Thread_Allocation.PNG
<http://qpid.2158936.n2.nabble.com/file/t396358/Sampler_Per_Thread_Allocation.PNG>
TestQpidRcvOnly.txt
<http://qpid.2158936.n2.nabble.com/file/t396358/TestQpidRcvOnly.txt>
Total_Memory.PNG
<http://qpid.2158936.n2.nabble.com/file/t396358/Total_Memory.PNG>
--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Out Of Memory error - Qpid not releasing memory in failed to
acknowledge scenario
Posted by Timothy Bish <ta...@gmail.com>.
On 2/7/20 11:57 AM, akabhishek1 wrote:
> Hi Tim,
>
> Thanks a lot for quick reply. Please find attached updated code. I am
> intentionally throwing exception after receiving the message. We could face
> this issue in production.
>
> We have implemented reliability pattern, message should be acknowledge only
> after successful transformation. Bussiness transformation/process could be
> fail continously due to unavailability of service.
>
> We are using *connection.createSession(false, 101)* , so only one message
> will be acknowledged at a time not prior messages.
Client individual ack mode operates in fundamentally the same manner as
Client Ack mode where failures due to exceptions in the asynchronous
path must be handled by a call to session recover to replay the message
and reprocess it, or close the session if you can't handle reprocessing
of failed messages. You can't just throw an exception in the async
MessageListener onMessage handler and hope for the best here.
>
> The given example is just to replicate Out of Memory error based on
> bussiness scenario.
>
> Please suggest, if i can do anything to avoid this error.
>
Don't throw exceptions from onMessage as the specification labels this a
client programming error.
> TestQpidRcvOnly.txt
> <http://qpid.2158936.n2.nabble.com/file/t396358/TestQpidRcvOnly.txt>
>
>
>
> --
> Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
--
Tim Bish
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Out Of Memory error - Qpid not releasing memory in failed to
acknowledge scenario
Posted by akabhishek1 <ma...@gmail.com>.
Hi Tim,
Thanks a lot for quick reply. Please find attached updated code. I am
intentionally throwing exception after receiving the message. We could face
this issue in production.
We have implemented reliability pattern, message should be acknowledge only
after successful transformation. Bussiness transformation/process could be
fail continously due to unavailability of service.
We are using *connection.createSession(false, 101)* , so only one message
will be acknowledged at a time not prior messages.
The given example is just to replicate Out of Memory error based on
bussiness scenario.
Please suggest, if i can do anything to avoid this error.
TestQpidRcvOnly.txt
<http://qpid.2158936.n2.nabble.com/file/t396358/TestQpidRcvOnly.txt>
--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Out Of Memory error - Qpid not releasing memory in failed to
acknowledge scenario
Posted by Timothy Bish <ta...@gmail.com>.
On 2/7/20 7:16 AM, akabhishek1 wrote:
> Hi Team,
>
> We were doing testing of Qpid JMS client for failed scenario. We are
> receiving message as client acknowledgment mode.
>
> Scenario - Receive message but do not acknowledge the message.
>
> Issue - Memory is not getting released by Qpid on failed Scenario
In client acknowledge mode it is your responsibility to acknowledge
messages when they have been processed. The mechanics of the JMS
consumer and session in client ack mode require that messages be
retained such that a call to session recover can replay the delivered
but unacknowledged messages so we cannot just release them on delivery.
Your sample is just receiving forever without any error conditions being
simulated or handled so the client will accumulate messages. This is
really just a misuse of the client, you can resolve it in the test by
periodically calling acknowledge on an incoming message and all previous
messages will be acknowledged and releases and you'd see heap usage
decrease as a result.
You mention that you are trying to test failed scenarios but the code
given doesn't seem to be expressing any sort of handling of same so you
might want to evaluate what you are trying to accomplish a bit more and
maybe ask for help here on that end.
>
> We could have this scenario in prodcution. We received the message, but
> failed to process that message, so there will no aknowledgment. As we are
> acknowledging only one success scenario.
>
> Please find attached sample TestQpidRcvOnly.java, Total_Memory.PNG,
> Sampler_Per_Thread_Allocation.PNG, Out_Of_Memory_Error.log .
>
> Steps to reproduce this issue -->
>
> 1. Provide Correct details of QUEUE_NAME, SBUS_NAME, USERNAME, USERNAME in
> TestQpidRcvOnly class
> 2. Publish 200000 message on QUEUE_NAME, from any utility
> 3. Start the TestQpidRcvOnly class and keep it running
> 4. Open JVisualVM and have a look on Heap memory
> 5. Wait for 20m, you can see memory consumption continously increasing
> 6. You can see Out Of Memory Error on exhaustion of defined Max heap Memory.
> Could be 1hr depends on Max Size
> 7. You can have a look on Sampler--> Memory -->Per Thread Allocation, you
> can observe that Qpid thread is consuming more that 98% memory.
>
> Consumption is OK but it is not releasing the memory, which is the cause for
> Out of Memory Error.
>
> Using -- Java 8, Azure ServiceBus, Qpid-jms-client-0.48
>
>
> Could you please take a look on this issue.
>
> Regards,
> Abhishek Kumar
>
> Out_Of_Memory_Error.log
> <http://qpid.2158936.n2.nabble.com/file/t396358/Out_Of_Memory_Error.log>
> Sampler_Per_Thread_Allocation.PNG
> <http://qpid.2158936.n2.nabble.com/file/t396358/Sampler_Per_Thread_Allocation.PNG>
> TestQpidRcvOnly.txt
> <http://qpid.2158936.n2.nabble.com/file/t396358/TestQpidRcvOnly.txt>
> Total_Memory.PNG
> <http://qpid.2158936.n2.nabble.com/file/t396358/Total_Memory.PNG>
>
>
>
> --
> Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
--
Tim Bish
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org