You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by pdudits <na...@dudits.net> on 2016/08/04 14:33:17 UTC
Messages piling up in FailoverTranport
Hello,
I'm facing a situation where client go OutOfMemory due to ever growing
transaction log in
FailoverTransport.stateTracker.connectionStates[one specific
connection].transactions.
The majority of recorded transactions contain three commands:
1. TransactionInfo BEGIN
2. The message the client sent as a response to a temp queue
3. TransactionInfo END
I upgraded broker to 5.13.4, as the symptomps on the broker corresponded to
https://issues.apache.org/jira/browse/AMQ-6194,
but that only made the error on the broker go away.
The client is an MDB with activemq-ra 5.13.3 running in Payara 4.1.161,
listening on a queue.
My question is what may be causing this behavior? Is it a bug?
The transaction is surely commited as these responses on temp queues are
delivered, so the broker must have received the commit command, it just
looks it didn't get passed back to the connection.
-----
---
Patrik Dudits
--
View this message in context: http://activemq.2283324.n4.nabble.com/Messages-piling-up-in-FailoverTranport-tp4715072.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Messages piling up in FailoverTranport
Posted by pdudits <na...@dudits.net>.
I concluded that it is a bug and filed
https://issues.apache.org/jira/browse/AMQ-6391.
-----
---
Patrik Dudits
--
View this message in context: http://activemq.2283324.n4.nabble.com/Messages-piling-up-in-FailoverTranport-tp4715072p4715258.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Messages piling up in FailoverTranport
Posted by pdudits <na...@dudits.net>.
Thank you, Martin,
this is indeed a result of heap dump analysis. Under normal conditions the
application
operates happilly with around 800MB of heap as can be seen in visualvm. In
these out
of memory situations, 1.2 - 1.6 GBs is allocated in that very object in the
description.
It is always this connection that is used to put responses on temp queue.
Other MDBs produce messages without FailoverTransport not cleaning commited
transactions.
I'll surely try to find out a minimal example. I think this is only MDB that
responds via
transactional connection pool to a temp queue. Other put messages to normal
queues,
topics, or use non-transactional pool for sending to temp queues.
-----
---
Patrik Dudits
--
View this message in context: http://activemq.2283324.n4.nabble.com/Messages-piling-up-in-FailoverTranport-tp4715072p4715080.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Messages piling up in FailoverTranport
Posted by Marcin Giedz <ma...@arise.pl>.
Sent from my BlackBerry 10 smartphone.
Original Message
From: Martin Herrman
Sent: Thursday, August 4, 2016 17:44
To: users@activemq.apache.org
Reply To: users@activemq.apache.org
Subject: Re: Messages piling up in FailoverTranport
Hi Patrik,
My approach would be to gather more information about the memory usage. A method is to enable JMX and use VisualVM to create the graphs. These links might help:
http://blog.payara.fish/troubleshooting-your-java-ee-applications
https://visualvm.java.net/monitor_tab.html
HTH,
Martin
Van: pdudits <na...@dudits.net>
Verzonden: donderdag 4 augustus 2016 16:33
Aan: users@activemq.apache.org
Onderwerp: Messages piling up in FailoverTranport
Hello,
I'm facing a situation where client go OutOfMemory due to ever growing
transaction log in
FailoverTransport.stateTracker.connectionStates[one specific
connection].transactions.
The majority of recorded transactions contain three commands:
1. TransactionInfo BEGIN
2. The message the client sent as a response to a temp queue
3. TransactionInfo END
I upgraded broker to 5.13.4, as the symptomps on the broker corresponded to
https://issues.apache.org/jira/browse/AMQ-6194,
but that only made the error on the broker go away.
The client is an MDB with activemq-ra 5.13.3 running in Payara 4.1.161,
listening on a queue.
My question is what may be causing this behavior? Is it a bug?
The transaction is surely commited as these responses on temp queues are
delivered, so the broker must have received the commit command, it just
looks it didn't get passed back to the connection.
Met vriendelijke groet/Kind regards,
Martin Herrman
Manager DevOps
[YMOR]<http://www.ymor.com>
Ptolemaeuslaan 70
3528 BP Utrecht
Mobiel: +31 6 10761936
Kantoor: +31(0)30 6021149
Email: mherrman@ymor.nl
[ITOA]
Van reactief naar preventief IT-beheer?
Download de whitepaper:
'Van achteruitkijkspiegel naar glazen bol met IT Operations Analytics’<http://www.ymor.com/whitepapers/>
This e-mail message and its attachments are subject to the disclaimer published on our website<https://www.ymor.com/files/2016/04/2016-Disclaimer-and-Security-EN.pdf>
Re: Messages piling up in FailoverTranport
Posted by Martin Herrman <mh...@ymor.nl>.
Hi Patrik,
My approach would be to gather more information about the memory usage. A method is to enable JMX and use VisualVM to create the graphs. These links might help:
http://blog.payara.fish/troubleshooting-your-java-ee-applications
https://visualvm.java.net/monitor_tab.html
HTH,
Martin
Van: pdudits <na...@dudits.net>
Verzonden: donderdag 4 augustus 2016 16:33
Aan: users@activemq.apache.org
Onderwerp: Messages piling up in FailoverTranport
Hello,
I'm facing a situation where client go OutOfMemory due to ever growing
transaction log in
FailoverTransport.stateTracker.connectionStates[one specific
connection].transactions.
The majority of recorded transactions contain three commands:
1. TransactionInfo BEGIN
2. The message the client sent as a response to a temp queue
3. TransactionInfo END
I upgraded broker to 5.13.4, as the symptomps on the broker corresponded to
https://issues.apache.org/jira/browse/AMQ-6194,
but that only made the error on the broker go away.
The client is an MDB with activemq-ra 5.13.3 running in Payara 4.1.161,
listening on a queue.
My question is what may be causing this behavior? Is it a bug?
The transaction is surely commited as these responses on temp queues are
delivered, so the broker must have received the commit command, it just
looks it didn't get passed back to the connection.
Met vriendelijke groet/Kind regards,
Martin Herrman
Manager DevOps
[YMOR]<http://www.ymor.com>
Ptolemaeuslaan 70
3528 BP Utrecht
Mobiel: +31 6 10761936
Kantoor: +31(0)30 6021149
Email: mherrman@ymor.nl
[ITOA]
Van reactief naar preventief IT-beheer?
Download de whitepaper:
'Van achteruitkijkspiegel naar glazen bol met IT Operations Analytics’<http://www.ymor.com/whitepapers/>
This e-mail message and its attachments are subject to the disclaimer published on our website<https://www.ymor.com/files/2016/04/2016-Disclaimer-and-Security-EN.pdf>