You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Justin Bertram (Jira)" <ji...@apache.org> on 2022/09/27 14:24:00 UTC
[jira] [Updated] (ARTEMIS-4018) Upgrade: Existing Messages not published and journal export runs out of memory
[ https://issues.apache.org/jira/browse/ARTEMIS-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Justin Bertram updated ARTEMIS-4018:
------------------------------------
Description:
This might be two separate bugs:
# After upgrading from version 2.22.0 to 2.25.0 we were facing a situation where messages, already queued or paged before the upgrade are not delivered to consumers. New messages, arriving at the broker after the upgrade were handled normally.
# To avoid the above situation on other nodes, we wanted to export the old journal and reimport it before restarting the upgraded version. The export however runs OOM:
{noformat}
/var/opt/apache-artemis/integrationbus>bin/artemis data exp --output /var/opt/apache-artemis/integrationbus/etc/data3.exp
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at io.netty.buffer.UnpooledByteBufAllocator.newHeapBuffer(UnpooledByteBufAllocator.java:82)
at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:169)
at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:160)
at io.netty.buffer.Unpooled.buffer(Unpooled.java:119)
at org.apache.activemq.artemis.core.message.impl.CoreMessage.initBuffer(CoreMessage.java:156)
at org.apache.activemq.artemis.core.message.impl.CoreMessage.reloadPersistence(CoreMessage.java:1252)
at org.apache.activemq.artemis.core.message.impl.CoreMessagePersister.decode(CoreMessagePersister.java:85)
at org.apache.activemq.artemis.core.message.impl.CoreMessagePersister.decode(CoreMessagePersister.java:28)
at org.apache.activemq.artemis.spi.core.protocol.MessagePersister.decode(MessagePersister.java:120)
at org.apache.activemq.artemis.core.persistence.impl.journal.DescribeJournal.newObjectEncoding(DescribeJournal.java:623)
at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.processMessageJournal(XmlDataExporter.java:218)
at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.writeXMLData(XmlDataExporter.java:150)
at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.writeOutput(XmlDataExporter.java:144)
at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.process(XmlDataExporter.java:133)
at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.execute(XmlDataExporter.java:100)
at org.apache.activemq.artemis.cli.Artemis.internalExecute(Artemis.java:160)
at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:108)
at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:135)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.activemq.artemis.boot.Artemis.execute(Artemis.java:134)
at org.apache.activemq.artemis.boot.Artemis.main(Artemis.java:50){noformat}
Note: We were attempting to export the old journal with the new broker version.
was:
This might be two separate bugs:
# After upgrading from version 2.22.0 to 2.25.0 we were facing a situation where messages, already queued or paged before the upgrade are not delivered to consumers. New messages, arriving at the broker after the upgrade were handled normally.
# To avoid the above situation on other nodes, we wanted to export the old journal and reimport it before restarting the upgraded version. The export however runs OOM:
{{/var/opt/apache-artemis/integrationbus>bin/artemis data exp --output /var/opt/apache-artemis/integrationbus/etc/data3.exp
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at io.netty.buffer.UnpooledByteBufAllocator.newHeapBuffer(UnpooledByteBufAllocator.java:82)
at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:169)
at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:160)
at io.netty.buffer.Unpooled.buffer(Unpooled.java:119)
at org.apache.activemq.artemis.core.message.impl.CoreMessage.initBuffer(CoreMessage.java:156)
at org.apache.activemq.artemis.core.message.impl.CoreMessage.reloadPersistence(CoreMessage.java:1252)
at org.apache.activemq.artemis.core.message.impl.CoreMessagePersister.decode(CoreMessagePersister.java:85)
at org.apache.activemq.artemis.core.message.impl.CoreMessagePersister.decode(CoreMessagePersister.java:28)
at org.apache.activemq.artemis.spi.core.protocol.MessagePersister.decode(MessagePersister.java:120)
at org.apache.activemq.artemis.core.persistence.impl.journal.DescribeJournal.newObjectEncoding(DescribeJournal.java:623)
at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.processMessageJournal(XmlDataExporter.java:218)
at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.writeXMLData(XmlDataExporter.java:150)
at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.writeOutput(XmlDataExporter.java:144)
at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.process(XmlDataExporter.java:133)
at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.execute(XmlDataExporter.java:100)
at org.apache.activemq.artemis.cli.Artemis.internalExecute(Artemis.java:160)
at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:108)
at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:135)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.activemq.artemis.boot.Artemis.execute(Artemis.java:134)
at org.apache.activemq.artemis.boot.Artemis.main(Artemis.java:50)}}
Note: We were attempting to export the old journal with the new broker version.
> Upgrade: Existing Messages not published and journal export runs out of memory
> -------------------------------------------------------------------------------
>
> Key: ARTEMIS-4018
> URL: https://issues.apache.org/jira/browse/ARTEMIS-4018
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Environment: SUSE Linux Enterprise Server 15 SP3
> JAVA_ARGS="-XX:+PrintClassHistogram -XX:+UseG1GC -XX:+UseStringDeduplication -Xms512M -Xmx4G -Dhawtio.disableProxy=true -Dhawtio.realm=activemq -Dhawtio.offline=true -Dhawtio.rolePrincipalClasses=org.apache.activemq.artemis.spi.core.security.jaas.RolePrincipal -Djolokia.policyLocation=${ARTEMIS_INSTANCE_ETC_URI}jolokia-access.xml "
> Reporter: Peter Machon
> Priority: Major
>
> This might be two separate bugs:
> # After upgrading from version 2.22.0 to 2.25.0 we were facing a situation where messages, already queued or paged before the upgrade are not delivered to consumers. New messages, arriving at the broker after the upgrade were handled normally.
> # To avoid the above situation on other nodes, we wanted to export the old journal and reimport it before restarting the upgraded version. The export however runs OOM:
> {noformat}
> /var/opt/apache-artemis/integrationbus>bin/artemis data exp --output /var/opt/apache-artemis/integrationbus/etc/data3.exp
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at io.netty.buffer.UnpooledByteBufAllocator.newHeapBuffer(UnpooledByteBufAllocator.java:82)
> at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:169)
> at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:160)
> at io.netty.buffer.Unpooled.buffer(Unpooled.java:119)
> at org.apache.activemq.artemis.core.message.impl.CoreMessage.initBuffer(CoreMessage.java:156)
> at org.apache.activemq.artemis.core.message.impl.CoreMessage.reloadPersistence(CoreMessage.java:1252)
> at org.apache.activemq.artemis.core.message.impl.CoreMessagePersister.decode(CoreMessagePersister.java:85)
> at org.apache.activemq.artemis.core.message.impl.CoreMessagePersister.decode(CoreMessagePersister.java:28)
> at org.apache.activemq.artemis.spi.core.protocol.MessagePersister.decode(MessagePersister.java:120)
> at org.apache.activemq.artemis.core.persistence.impl.journal.DescribeJournal.newObjectEncoding(DescribeJournal.java:623)
> at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.processMessageJournal(XmlDataExporter.java:218)
> at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.writeXMLData(XmlDataExporter.java:150)
> at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.writeOutput(XmlDataExporter.java:144)
> at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.process(XmlDataExporter.java:133)
> at org.apache.activemq.artemis.cli.commands.tools.xml.XmlDataExporter.execute(XmlDataExporter.java:100)
> at org.apache.activemq.artemis.cli.Artemis.internalExecute(Artemis.java:160)
> at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:108)
> at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:135)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at org.apache.activemq.artemis.boot.Artemis.execute(Artemis.java:134)
> at org.apache.activemq.artemis.boot.Artemis.main(Artemis.java:50){noformat}
> Note: We were attempting to export the old journal with the new broker version.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)