You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Vesa Koskela (Jira)" <ji...@apache.org> on 2020/08/13 10:47:00 UTC
[jira] [Updated] (ARTEMIS-2879) Artemis unable to reconnect to
client after client restart
[ https://issues.apache.org/jira/browse/ARTEMIS-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vesa Koskela updated ARTEMIS-2879:
----------------------------------
Description:
We recently attempted to upgrade our Artemis from version 2.11.0 to 2.14.0 in our development environment. Our current CI/CD pipeline installs the nightly build to this environment, which leads to all Artemis clients being updated and restarted. We noticed that after the restart the client is unable to connect to Artemis.
Below are some snippets of what we found from Artemis logs.
Right after the nightly build was installed:
{code}
2020-08-12 02:03:45,728 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 07ba2a53-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,800 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 07ba2a53-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,800 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 07ba2a54-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,801 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 07ba2a54-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,866 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 28d62691-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,866 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 28d62691-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,867 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 28d64da2-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,867 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 28d64da2-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,867 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 2eaa760c-db93-11ea-bd51-0021f6976e26
{code}
This continues for 178 lines. Then we get an exception:
{code}
2020-08-12 02:03:46,138 ERROR [org.apache.activemq.artemis.core.journal.impl.JournalImpl] appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id 8233509: java.lang.IllegalStateException: Cannot find tx with id 8233509
at org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451) [artemis-journal-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.14.0.jar:2.14.0]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [java.base:]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [java.base:]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.14.0.jar:2.14.0]
{code}
Then we get a couple of {{WARN [org.apache.activemq.artemis.core.server] AMQ222104: IO Error completing the transaction, code = 6, message = Cannot find tx with id 8233509}} lines, followed up by two exceptions:
{code}
2020-08-12 02:03:46,216 ERROR [org.apache.activemq.artemis.core.journal.impl.JournalImpl] appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id 8233390: java.lang.IllegalStateException: Cannot find tx with id 8233390
at org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451) [artemis-journal-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.14.0.jar:2.14.0]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [java.base:]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [java.base:]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.14.0.jar:2.14.0]
2020-08-12 02:03:46,220 ERROR [org.apache.activemq.artemis.core.journal.impl.JournalImpl] appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id 8232236: java.lang.IllegalStateException: Cannot find tx with id 8232236
at org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451) [artemis-journal-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.14.0.jar:2.14.0]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [java.base:]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [java.base:]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.14.0.jar:2.14.0]
{code}
After that we get some more failed client connections, resource cleaning and now also some failed reattach requests.
{code}
2020-08-12 02:03:54,000 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 0fcaadee-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222104: IO Error completing the transaction, code = 6, message = Cannot find tx with id 8232236
2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 0fcaadee-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 0fcafc0f-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 0fcafc0f-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:54,009 WARN [org.apache.activemq.artemis.core.server] AMQ222066: Reattach request from /<server-ip>:45829 failed as there is no confirmationWindowSize configured, which may be ok for your system
2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 12d2bde3-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 12d2bde3-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 12e31194-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222104: IO Error completing the transaction, code = 6, message = Cannot find tx with id 8232236
2020-08-12 02:03:59,119 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 12e31194-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:59,119 WARN [org.apache.activemq.artemis.core.server] AMQ222066: Reattach request from /<server-ip>:46129 failed as there is no confirmationWindowSize configured, which may be ok for your system
{code}
These resource clearings, failed reattach requests, client connection failures and occasional exceptions continue until Artemis is restarted. After that Artemis works until the next nightly build is installed.
We rolled back to 2.11.0 and everything works again like it should be. So the problem seems to be mainly on how 2.14.0 handles reconnections after client is reinstalled. The client applications with these problems are using artemis client version 2.10.1.
was:
We recently attempted to upgrade our Artemis from version 2.11.0 to 2.14.0 in our development environment. Our current CI/CD pipeline installs the nightly build to this environment, which leads to all Artemis clients being updated and restarted. We noticed that after the restart the client is unable to connect to Artemis.
Below are some snippets of what we found from Artemis logs.
Right after the nightly build was installed:
{code}
2020-08-12 02:03:45,728 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 07ba2a53-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,800 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 07ba2a53-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,800 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 07ba2a54-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,801 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 07ba2a54-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,866 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 28d62691-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,866 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 28d62691-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,867 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 28d64da2-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,867 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 28d64da2-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,867 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 2eaa760c-db93-11ea-bd51-0021f6976e26
{code}
This continues for 178 lines. Then we get an exception:
{code}
2020-08-12 02:03:46,138 ERROR [org.apache.activemq.artemis.core.journal.impl.JournalImpl] appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id 8233509: java.lang.IllegalStateException: Cannot find tx with id 8233509
at org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451) [artemis-journal-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.14.0.jar:2.14.0]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [java.base:]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [java.base:]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.14.0.jar:2.14.0]
{code}
Then we get a couple of {{WARN [org.apache.activemq.artemis.core.server] AMQ222104: IO Error completing the transaction, code = 6, message = Cannot find tx with id 8233509}} lines, followed up by two exceptions:
{code}
2020-08-12 02:03:46,216 ERROR [org.apache.activemq.artemis.core.journal.impl.JournalImpl] appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id 8233390: java.lang.IllegalStateException: Cannot find tx with id 8233390
at org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451) [artemis-journal-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.14.0.jar:2.14.0]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [java.base:]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [java.base:]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.14.0.jar:2.14.0]
2020-08-12 02:03:46,220 ERROR [org.apache.activemq.artemis.core.journal.impl.JournalImpl] appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id 8232236: java.lang.IllegalStateException: Cannot find tx with id 8232236
at org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451) [artemis-journal-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.14.0.jar:2.14.0]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [java.base:]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [java.base:]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.14.0.jar:2.14.0]
{code}
After that we get some more failed client connections, resource cleaning and now also some failed reattach requests.
{code}
2020-08-12 02:03:54,000 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 0fcaadee-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222104: IO Error completing the transaction, code = 6, message = Cannot find tx with id 8232236
2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 0fcaadee-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 0fcafc0f-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 0fcafc0f-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:54,009 WARN [org.apache.activemq.artemis.core.server] AMQ222066: Reattach request from /172.30.41.37:45829 failed as there is no confirmationWindowSize configured, which may be ok for your system
2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 12d2bde3-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 12d2bde3-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 12e31194-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222104: IO Error completing the transaction, code = 6, message = Cannot find tx with id 8232236
2020-08-12 02:03:59,119 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 12e31194-dc40-11ea-b431-5254000e9dfd
2020-08-12 02:03:59,119 WARN [org.apache.activemq.artemis.core.server] AMQ222066: Reattach request from /172.30.41.37:46129 failed as there is no confirmationWindowSize configured, which may be ok for your system
{code}
These resource clearings, failed reattach requests, client connection failures and occasional exceptions continue until Artemis is restarted. After that Artemis works until the next nightly build is installed.
We rolled back to 2.11.0 and everything works again like it should be. So the problem seems to be mainly on how 2.14.0 handles reconnections after client is reinstalled. The client applications with these problems are using artemis client version 2.10.1.
> Artemis unable to reconnect to client after client restart
> ----------------------------------------------------------
>
> Key: ARTEMIS-2879
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2879
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Affects Versions: 2.14.0
> Reporter: Vesa Koskela
> Priority: Major
>
> We recently attempted to upgrade our Artemis from version 2.11.0 to 2.14.0 in our development environment. Our current CI/CD pipeline installs the nightly build to this environment, which leads to all Artemis clients being updated and restarted. We noticed that after the restart the client is unable to connect to Artemis.
> Below are some snippets of what we found from Artemis logs.
> Right after the nightly build was installed:
> {code}
> 2020-08-12 02:03:45,728 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 07ba2a53-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,800 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 07ba2a53-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,800 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 07ba2a54-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,801 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 07ba2a54-dc40-11ea-827f-0021f6976e26 2020-08-12 02:03:45,866 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 28d62691-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,866 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 28d62691-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,867 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 28d64da2-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,867 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 28d64da2-db93-11ea-bd51-0021f6976e26 2020-08-12 02:03:45,867 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 2eaa760c-db93-11ea-bd51-0021f6976e26
> {code}
> This continues for 178 lines. Then we get an exception:
> {code}
> 2020-08-12 02:03:46,138 ERROR [org.apache.activemq.artemis.core.journal.impl.JournalImpl] appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id 8233509: java.lang.IllegalStateException: Cannot find tx with id 8233509
> at org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451) [artemis-journal-2.14.0.jar:2.14.0]
> at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.14.0.jar:2.14.0]
> at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.14.0.jar:2.14.0]
> at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.14.0.jar:2.14.0]
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [java.base:]
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [java.base:]
> at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.14.0.jar:2.14.0]
> {code}
> Then we get a couple of {{WARN [org.apache.activemq.artemis.core.server] AMQ222104: IO Error completing the transaction, code = 6, message = Cannot find tx with id 8233509}} lines, followed up by two exceptions:
> {code}
> 2020-08-12 02:03:46,216 ERROR [org.apache.activemq.artemis.core.journal.impl.JournalImpl] appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id 8233390: java.lang.IllegalStateException: Cannot find tx with id 8233390
> at org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451) [artemis-journal-2.14.0.jar:2.14.0]
> at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.14.0.jar:2.14.0]
> at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.14.0.jar:2.14.0]
> at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.14.0.jar:2.14.0]
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [java.base:]
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [java.base:]
> at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.14.0.jar:2.14.0]
> 2020-08-12 02:03:46,220 ERROR [org.apache.activemq.artemis.core.journal.impl.JournalImpl] appendRollbackRecord:java.lang.IllegalStateException: Cannot find tx with id 8232236: java.lang.IllegalStateException: Cannot find tx with id 8232236
> at org.apache.activemq.artemis.core.journal.impl.JournalImpl$10.run(JournalImpl.java:1451) [artemis-journal-2.14.0.jar:2.14.0]
> at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.14.0.jar:2.14.0]
> at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.14.0.jar:2.14.0]
> at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.14.0.jar:2.14.0]
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [java.base:]
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [java.base:]
> at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.14.0.jar:2.14.0]
> {code}
> After that we get some more failed client connections, resource cleaning and now also some failed reattach requests.
> {code}
> 2020-08-12 02:03:54,000 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 0fcaadee-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222104: IO Error completing the transaction, code = 6, message = Cannot find tx with id 8232236
> 2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 0fcaadee-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 0fcafc0f-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:54,001 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 0fcafc0f-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:54,009 WARN [org.apache.activemq.artemis.core.server] AMQ222066: Reattach request from /<server-ip>:45829 failed as there is no confirmationWindowSize configured, which may be ok for your system
> 2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 12d2bde3-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 12d2bde3-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session 12e31194-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:59,118 WARN [org.apache.activemq.artemis.core.server] AMQ222104: IO Error completing the transaction, code = 6, message = Cannot find tx with id 8232236
> 2020-08-12 02:03:59,119 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session 12e31194-dc40-11ea-b431-5254000e9dfd
> 2020-08-12 02:03:59,119 WARN [org.apache.activemq.artemis.core.server] AMQ222066: Reattach request from /<server-ip>:46129 failed as there is no confirmationWindowSize configured, which may be ok for your system
> {code}
> These resource clearings, failed reattach requests, client connection failures and occasional exceptions continue until Artemis is restarted. After that Artemis works until the next nightly build is installed.
> We rolled back to 2.11.0 and everything works again like it should be. So the problem seems to be mainly on how 2.14.0 handles reconnections after client is reinstalled. The client applications with these problems are using artemis client version 2.10.1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)