You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Tamas Szabo (JIRA)" <ji...@apache.org> on 2017/02/23 16:51:44 UTC
[jira] [Commented] (AMQ-6609) A lot async error and null pointer
exception
[ https://issues.apache.org/jira/browse/AMQ-6609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880825#comment-15880825 ]
Tamas Szabo commented on AMQ-6609:
----------------------------------
Thanks I'll try to update it. So after update I should report it anywhere else?
> A lot async error and null pointer exception
> --------------------------------------------
>
> Key: AMQ-6609
> URL: https://issues.apache.org/jira/browse/AMQ-6609
> Project: ActiveMQ
> Issue Type: Bug
> Components: JMX, networkbridge, Transport
> Affects Versions: 5.5.0
> Environment: 2 virtual machines, in same network.
> Having 2 AMQ server instance, with a network connection. HA - Active-Active mode.
> *CONFIG#1*
> {code:xml}
> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:amq="http://activemq.apache.org/schema/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
> <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
> <property name="locations">
> <value>file:${activemq.base}/conf/credentials.properties</value>
> </property>
> </bean>
> <broker xmlns="http://activemq.apache.org/schema/core" brokerName="xx1" dataDirectory="${activemq.base}/data" destroyApplicationContextOnStop="true" deleteAllMessagesOnStartup="true" networkConnectorStartAsync="true" enableStatistics="false" schedulerSupport="false" useJmx="true">
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="true" memoryLimit="5mb">
> <pendingSubscriberPolicy>
> <fileCursor />
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="true" memoryLimit="5mb" optimizedDispatch="true">
> <deadLetterStrategy>
> <sharedDeadLetterStrategy processExpired="false" />
> </deadLetterStrategy>
> <pendingQueuePolicy>
> <fileQueueCursor />
> </pendingQueuePolicy>
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
> <managementContext>
> <managementContext createConnector="false" />
> </managementContext>
> <persistenceAdapter>
> <kahaDB directory="${activemq.base}/data/kahadb" enableIndexWriteAsync="true" />
> </persistenceAdapter>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage limit="500 mb" />
> </memoryUsage>
> <storeUsage>
> <storeUsage limit="1 gb" />
> </storeUsage>
> <tempUsage>
> <tempUsage limit="1 gb" />
> </tempUsage>
> </systemUsage>
> </systemUsage>
> <transportConnectors>
> <transportConnector name="openwire" uri="nio://0.0.0.0:5672?keepAlive=true&soTimeout=60000&wireFormat.maxInactivityDurationInitalDelay=8000&wireFormat.maxInactivityDuration=19000" />
> </transportConnectors>
> </broker>
> <import resource="jetty.xml" />
> </beans>
> {code}
> *CONFIG#2:*
> {code:xml}
> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:amq="http://activemq.apache.org/schema/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
> <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
> <property name="locations">
> <value>file:${activemq.base}/conf/credentials.properties</value>
> </property>
> </bean>
> <broker xmlns="http://activemq.apache.org/schema/core" brokerName="xx2" dataDirectory="${activemq.base}/data" destroyApplicationContextOnStop="true" deleteAllMessagesOnStartup="true" networkConnectorStartAsync="true" enableStatistics="false" schedulerSupport="false" useJmx="true">
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="true" memoryLimit="5mb">
> <pendingSubscriberPolicy>
> <fileCursor />
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="true" memoryLimit="5mb" optimizedDispatch="true">
> <deadLetterStrategy>
> <sharedDeadLetterStrategy processExpired="false" />
> </deadLetterStrategy>
> <pendingQueuePolicy>
> <fileQueueCursor />
> </pendingQueuePolicy>
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
> <managementContext>
> <managementContext createConnector="false" />
> </managementContext>
> <networkConnectors>
> <networkConnector uri="static:(tcp://xx1:5672)" duplex="true" conduitSubscriptions="true" decreaseNetworkConsumerPriority="true" />
> </networkConnectors>
> <persistenceAdapter>
> <kahaDB directory="${activemq.base}/data/kahadb" enableIndexWriteAsync="true" />
> </persistenceAdapter>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage limit="500 mb" />
> </memoryUsage>
> <storeUsage>
> <storeUsage limit="1 gb" />
> </storeUsage>
> <tempUsage>
> <tempUsage limit="1 gb" />
> </tempUsage>
> </systemUsage>
> </systemUsage>
> <transportConnectors>
> <transportConnector name="openwire" uri="nio://0.0.0.0:5672?keepAlive=true&soTimeout=60000&wireFormat.maxInactivityDurationInitalDelay=8000&wireFormat.maxInactivityDuration=19000" />
> </transportConnectors>
> </broker>
> <import resource="jetty.xml" />
> </beans>
> {code}
> Reporter: Tamas Szabo
> Labels: windows
>
> We have a production system installed, and on peaks we get a lot error because of undelivered messages.
> We have a java application which usually does the logic, and a c# application which makes requests to the java app.
> The error is customer sees a lot slowness in their system because of these errors.
> Errors are following:
> *Error-1*
> *XX-1 LOG*
> {quote}
> 2017-02-21 08:07:06,075 | WARN | Network connection between vm://xx-2#68 and tcp:///xx-2:5672 shutdown due to a remote error: java.net.SocketException: Software caused connection abort: recv failed | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ Transport: tcp:///xx-2:5672
> 2017-02-21 08:07:06,075 | INFO | Network connection between vm://xx-2#68 and tcp:///xx-2:5672 shutdown due to a local error: java.net.SocketException: Software caused connection abort: socket write error | org.apache.activemq.network.DemandForwardingBridgeSupport | BrokerService[xx-2] Task-472152
> {quote}
> *XX-2 LOG*
> {quote}
> 2017-02-21 08:07:05,476 | INFO | Network connection between vm://xx-1#68 and tcp:///xx-1:50234 shutdown due to a local error: javax.jms.JMSException: The destination temp-queue://ID:XYZA1234-65130-636232610113011707-1:0:2 does not exist. | org.apache.activemq.network.DemandForwardingBridgeSupport | BrokerService[xx-1] Task-592541
> 2017-02-21 08:07:10,500 | INFO | The connection to 'null' is taking a long time to shutdown. | org.apache.activemq.broker.TransportConnection | ActiveMQ Task-15133
> 2017-02-21 08:07:15,508 | INFO | The connection to 'null' is taking a long time to shutdown. | org.apache.activemq.broker.TransportConnection | ActiveMQ Task-15133
> 2017-02-21 08:07:20,520 | INFO | The connection to 'null' is taking a long time to shutdown. | org.apache.activemq.broker.TransportConnection | ActiveMQ Task-15133
> 2017-02-21 08:07:25,528 | INFO | The connection to 'null' is taking a long time to shutdown. | org.apache.activemq.broker.TransportConnection | ActiveMQ Task-15133
> 2017-02-21 08:07:30,536 | INFO | The connection to 'null' is taking a long time to shutdown. | org.apache.activemq.broker.TransportConnection | ActiveMQ Task-15133
> 2017-02-21 08:07:30,941 | INFO | Connector vm://xx-1 Stopped | org.apache.activemq.broker.TransportConnector | ActiveMQ Task-15134
> 2017-02-21 08:07:30,941 | INFO | xx-1 bridge to xx-2 stopped | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ Task-15134
> 2017-02-21 08:08:11,051 | INFO | Connector vm://xx-1 Started | org.apache.activemq.broker.TransportConnector | ActiveMQ NIO Worker
> 2017-02-21 08:08:11,083 | INFO | Started responder end of duplex bridge NC@ID:XX-2-55636-1486253750115-0:1 | org.apache.activemq.broker.TransportConnection | ActiveMQ NIO Worker
> 2017-02-21 08:08:11,083 | INFO | Network connection between vm://xx-1#70 and tcp:///xx-1:50249(xx-2) has been established. | org.apache.activemq.network.DemandForwardingBridgeSupport | StartLocalBridge: localBroker=vm://xx-1#70
> 2017-02-21 08:32:23,098 | INFO | Transport failed: java.io.IOException: An existing connection was forcibly closed by the remote host | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ NIO Worker
> 2017-02-21 08:39:44,635 | INFO | Transport failed: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too (>19000) long: /11.111.111.11:64200 | org.apache.activemq.broker.TransportConnection.Transport | InactivityMonitor Async Task: java.util.concurrent.ThreadPoolExecutor$Worker@37d226bd
> 2017-02-21 08:47:31,816 | INFO | Transport failed: java.io.IOException: An existing connection was forcibly closed by the remote host | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ NIO Worker
> 2017-02-21 08:51:17,393 | INFO | Transport failed: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too (>19000) long: /11.111.111.12:53436 | org.apache.activemq.broker.TransportConnection.Transport | InactivityMonitor Async Task: java.util.concurrent.ThreadPoolExecutor$Worker@511249ed
> 2017-02-21 08:51:47,940 | INFO | Transport failed: java.io.IOException: An existing connection was forcibly closed by the remote host | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ NIO Worker
> 2017-02-21 08:59:18,723 | INFO | Transport failed: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too (>19000) long: /11.111.111.14:51599 | org.apache.activemq.broker.TransportConnection.Transport | InactivityMonitor Async Task: java.util.concurrent.ThreadPoolExecutor$Worker@3d79ba00
> 2017-02-21 09:21:23,154 | INFO | Transport failed: java.io.IOException: An existing connection was forcibly closed by the remote host | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ NIO Worker
> 2017-02-21 09:28:38,819 | INFO | Transport failed: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too (>19000) long: /11.111.111.14:49185 | org.apache.activemq.broker.TransportConnection.Transport | InactivityMonitor Async Task: java.util.concurrent.ThreadPoolExecutor$Worker@6a3566bf
> 2017-02-21 09:33:57,336 | INFO | Transport failed: java.io.IOException: An existing connection was forcibly closed by the remote host | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ NIO Worker
> 2017-02-21 09:49:52,090 | INFO | Transport failed: java.io.IOException: An existing connection was forcibly closed by the remote host | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ NIO Worker
> 2017-02-21 09:54:22,246 | INFO | Transport failed: java.io.IOException: An existing connection was forcibly closed by the remote host | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ NIO Worker
> 2017-02-21 10:12:02,324 | INFO | Transport failed: java.io.IOException: An existing connection was forcibly closed by the remote host | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ NIO Worker
> 2017-02-21 11:14:57,879 | INFO | ActiveMQ Message Broker (xx-1, ID:XX-1-49435-1486253561864-0:1) is shutting down | org.apache.activemq.broker.BrokerService | Thread-24
> 2017-02-21 11:14:57,879 | INFO | ActiveMQ Message Broker (xx-1, ID:XX-1-49435-1486253561864-0:1) is shutting down | org.apache.activemq.broker.BrokerService | ActiveMQ ShutdownHook
> {quote}
> *Error-2*
> We have a lot of following errors (daily hundreds or more.)
> {quote}
> 2017-02-21 08:07:05,476 | WARN | Async error occurred: javax.jms.JMSException: The destination temp-queue://ID:XYZA1234-65130-123123123123123123-1:0:2 does not exist. | org.apache.activemq.broker.TransportConnection.Service | ActiveMQ NIO Worker
> javax.jms.JMSException: The destination temp-queue://ID:XYZA1234-65130-123123123123123123-1:0:2 does not exist.
> at org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:135)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)