You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Marcin Giedz <ma...@arise.pl> on 2017/04/19 14:37:34 UTC
Occasionally EOF on connection
Hello all,
in our framework we use AMQ 5.14.1 and CMS version 3.9.4. From time to time we observe connection drops on client's side like the one below:
20170418 10:00:41.902 CRITICAL: void CMSConnectionManager::sltConnectionError(CMSExceptionListener::errorType, QString) "DataInputStream::readLong - Reached EOF" "tcp://127.0.0.1:61616?connection.useAsyncSend=true&soKeepAlive=true&wireFormat.MaxInactivityDuration=240000"
in the same time on Broker side this is reported in log file:
2017-04-18 10:00:29,358 | WARN | Transport Connection to: tcp://127.0.0.1:48612 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ NIO Worker 8
questions:
1. is there any way to log more detailed information when EOF occurs ? I mean we don;t want to flood log file with general DEBUG messages but rather TransportConnection only to figure out what;s going on when EOF appears ?
2. perhaps we do have something wrong with broker/client settings ?
broker (part):
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}" useJmx="true" advisorySupport="true" persistent="true" schedulePeriodForDestinationPurge="10000" networkConnectorStartAsync="true" allowTempAutoCreationOnSend="true" deleteAllMessagesOnStartup="false" >
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="false" >
<pendingSubscriberPolicy>
<vmCursor/>
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="false" >
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
</policyEntry>
<policyEntry tempTopic="true" producerFlowControl="false" >
<pendingSubscriberPolicy>
<vmCursor/>
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry tempQueue="true" producerFlowControl="false" gcInactiveDestinations="true" inactiveTimoutBeforeGC="60000">
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<persistenceAdapter>
<levelDB directory="${activemq.data}/leveldb" logSize="107374182"/>
</persistenceAdapter>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage percentOfJvmHeap="80" />
</memoryUsage>
<storeUsage>
<storeUsage limit="20 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="10 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<transportConnector name="openwire" uri="nio://0.0.0.0:61616?transport.maximumConnections=1000&wireFormat.maxFrameSize=314572800&transport.keepAlive=true&wireFormat.maxInactivityDuration=240000"/>
</transportConnectors>
client:
[activemq]
URL="tcp://127.0.0.1:61616?connection.useAsyncSend=true&soKeepAlive=true&wireFormat.MaxInactivityDuration=240000"
Thx
Marcin
Re: Occasionally EOF on connection
Posted by Marcin Giedz <ma...@arise.pl>.
Tim how can I provide with more detailed logs? I can try and adjust broker side to have more debugs but client is unlikely to extend its messages
Thx
Marcin
Od: "Timothy Bish" <ta...@gmail.com>
Do: "users" <us...@activemq.apache.org>
Wysłane: środa, 26 kwiecień 2017 22:16:34
Temat: Re: Occasionally EOF on connection
On 04/26/2017 04:00 PM, Tim Bain wrote:
> Your right, a connection to localhost should be rock-solid. Sorry I missed
> that.
>
> Is there any possibility that a firewall (either OS or application) might
> be interfering with your connections? I feel like I'm grasping at straws a
> little bit, but I can't explain EOFs on both sides of the connection.
>
> The other variable here is that you're using the CMS client library (which
> I have no experience with and which isn't the subject of most posts on this
> list), and maybe the EOF it's showing doesn't mean the same thing as an
> EOFException would in the Java client library. @Tim Bish, do you see
> anything in the described behavior that would be specific to the CMS client
> and that might explain what's going on?
With the little information provided all I can say is that it appears
the connection was forcibly broken on both ends and the client and
broker reacted accordingly by issuing an EOFException. Without more
debugging there's really no why to know for sure what is going on.
>
> Tim
>
> On Apr 20, 2017 8:58 PM, "Marcin Giedz" <ma...@arise.pl> wrote:
>
>> Hi Tim,
>>
>> the main issue in this case is that this is over localhost connection -
>> URL contains 127.0.0.1 ;) we've got plenty of AMQ clients sitting on the
>> same host AMQ broker is running.
>>
>> wireshark is not a problem at all but I wonder how to "narrow" its
>> filtering. Perhaps I can also change log4j in AMQ broker to log more
>> details on transport connection level but I can't find how to do this
>> without global DEBUG/TRACE option ?
>>
>> Thx
>> Marcin
>>
>>
>> Od: "Tim Bain" <tb...@alumni.duke.edu>
>> Do: "users" <us...@activemq.apache.org>
>> Wysłane: czwartek, 20 kwiecień 2017 7:04:07
>> Temat: Re: Occasionally EOF on connection
>>
>> Nothing in your configuration looks obviously wrong to me.
>>
>> Is the connection between your broker and the clients that are having
>> problems unstable during the period of time that these problems are
>> occurring? Simultaneous EOFs on both ends of a socket connection makes me
>> think of packet loss or network disconnection; could that be what's going
>> on here?
>>
>> If you're allowed to use it in your environment, Wireshark could help you
>> figure out whether the problem is a failure of packets to be sent (e.g. due
>> to network problems) or if one side or the other is choosing to close the
>> connection.
>>
>> Tim
>>
>> On Apr 19, 2017 8:37 AM, "Marcin Giedz" <ma...@arise.pl> wrote:
>>
>>> Hello all,
>>>
>>> in our framework we use AMQ 5.14.1 and CMS version 3.9.4. From time to
>>> time we observe connection drops on client's side like the one below:
>>>
>>> 20170418 10:00:41.902 CRITICAL: void CMSConnectionManager::
>>> sltConnectionError(CMSExceptionListener::errorType, QString)
>>> "DataInputStream::readLong - Reached EOF" "tcp://127.0.0.1:61616?
>>> connection.useAsyncSend=true&soKeepAlive=true&wireFormat.
>>> MaxInactivityDuration=240000"
>>>
>>> in the same time on Broker side this is reported in log file:
>>>
>>> 2017-04-18 10:00:29,358 | WARN | Transport Connection to: tcp://
>>> 127.0.0.1:48612 failed: java.io.EOFException |
>> org.apache.activemq.broker.TransportConnection.Transport
>>> | ActiveMQ NIO Worker 8
>>>
>>> questions:
>>>
>>> 1. is there any way to log more detailed information when EOF occurs ? I
>>> mean we don;t want to flood log file with general DEBUG messages but
>> rather
>>> TransportConnection only to figure out what;s going on when EOF appears ?
>>> 2. perhaps we do have something wrong with broker/client settings ?
>>>
>>> broker (part):
>>> <broker xmlns="http://activemq.apache.org/schema/core"
>>> brokerName="localhost" dataDirectory="${activemq.data}" useJmx="true"
>>> advisorySupport="true" persistent="true" schedulePeriodForDestinationPu
>> rge="10000"
>>> networkConnectorStartAsync="true" allowTempAutoCreationOnSend="true"
>>> deleteAllMessagesOnStartup="false" >
>>>
>>> <destinationPolicy>
>>> <policyMap>
>>> <policyEntries>
>>> <policyEntry topic=">" producerFlowControl="false" >
>>> <pendingSubscriberPolicy>
>>> <vmCursor/>
>>> </pendingSubscriberPolicy>
>>> </policyEntry>
>>> <policyEntry queue=">" producerFlowControl="false" >
>>> <pendingQueuePolicy>
>>> <vmQueueCursor/>
>>> </pendingQueuePolicy>
>>> </policyEntry>
>>>
>>> <policyEntry tempTopic="true" producerFlowControl="false" >
>>> <pendingSubscriberPolicy>
>>> <vmCursor/>
>>> </pendingSubscriberPolicy>
>>> </policyEntry>
>>> <policyEntry tempQueue="true" producerFlowControl="false"
>>> gcInactiveDestinations="true" inactiveTimoutBeforeGC="60000">
>>> <pendingQueuePolicy>
>>> <vmQueueCursor/>
>>> </pendingQueuePolicy>
>>> </policyEntry>
>>>
>>> </policyEntries>
>>> </policyMap>
>>> </destinationPolicy>
>>>
>>> <persistenceAdapter>
>>> <levelDB directory="${activemq.data}/leveldb" logSize="107374182"/>
>>> </persistenceAdapter>
>>>
>>>
>>> <systemUsage>
>>> <systemUsage>
>>> <memoryUsage>
>>> <memoryUsage percentOfJvmHeap="80" />
>>> </memoryUsage>
>>> <storeUsage>
>>> <storeUsage limit="20 gb"/>
>>> </storeUsage>
>>> <tempUsage>
>>> <tempUsage limit="10 gb"/>
>>> </tempUsage>
>>> </systemUsage>
>>> </systemUsage>
>>>
>>> <transportConnectors>
>>> <transportConnector name="openwire" uri="nio://0.0.0.0:61616?
>>> transport.maximumConnections=1000&wireFormat.
>>> maxFrameSize=314572800&transport.keepAlive=true&wireFormat.
>>> maxInactivityDuration=240000"/>
>>> </transportConnectors>
>>>
>>> client:
>>> [activemq]
>>> URL="tcp://127.0.0.1:61616?connection.useAsyncSend=true&
>>> soKeepAlive=true&wireFormat.MaxInactivityDuration=240000"
>>>
>>> Thx
>>> Marcin
>>>
>> -
>>
Re: Occasionally EOF on connection
Posted by Timothy Bish <ta...@gmail.com>.
On 04/26/2017 04:00 PM, Tim Bain wrote:
> Your right, a connection to localhost should be rock-solid. Sorry I missed
> that.
>
> Is there any possibility that a firewall (either OS or application) might
> be interfering with your connections? I feel like I'm grasping at straws a
> little bit, but I can't explain EOFs on both sides of the connection.
>
> The other variable here is that you're using the CMS client library (which
> I have no experience with and which isn't the subject of most posts on this
> list), and maybe the EOF it's showing doesn't mean the same thing as an
> EOFException would in the Java client library. @Tim Bish, do you see
> anything in the described behavior that would be specific to the CMS client
> and that might explain what's going on?
With the little information provided all I can say is that it appears
the connection was forcibly broken on both ends and the client and
broker reacted accordingly by issuing an EOFException. Without more
debugging there's really no why to know for sure what is going on.
>
> Tim
>
> On Apr 20, 2017 8:58 PM, "Marcin Giedz" <ma...@arise.pl> wrote:
>
>> Hi Tim,
>>
>> the main issue in this case is that this is over localhost connection -
>> URL contains 127.0.0.1 ;) we've got plenty of AMQ clients sitting on the
>> same host AMQ broker is running.
>>
>> wireshark is not a problem at all but I wonder how to "narrow" its
>> filtering. Perhaps I can also change log4j in AMQ broker to log more
>> details on transport connection level but I can't find how to do this
>> without global DEBUG/TRACE option ?
>>
>> Thx
>> Marcin
>>
>>
>> Od: "Tim Bain" <tb...@alumni.duke.edu>
>> Do: "users" <us...@activemq.apache.org>
>> Wys\u0142ane: czwartek, 20 kwiecie\u0144 2017 7:04:07
>> Temat: Re: Occasionally EOF on connection
>>
>> Nothing in your configuration looks obviously wrong to me.
>>
>> Is the connection between your broker and the clients that are having
>> problems unstable during the period of time that these problems are
>> occurring? Simultaneous EOFs on both ends of a socket connection makes me
>> think of packet loss or network disconnection; could that be what's going
>> on here?
>>
>> If you're allowed to use it in your environment, Wireshark could help you
>> figure out whether the problem is a failure of packets to be sent (e.g. due
>> to network problems) or if one side or the other is choosing to close the
>> connection.
>>
>> Tim
>>
>> On Apr 19, 2017 8:37 AM, "Marcin Giedz" <ma...@arise.pl> wrote:
>>
>>> Hello all,
>>>
>>> in our framework we use AMQ 5.14.1 and CMS version 3.9.4. From time to
>>> time we observe connection drops on client's side like the one below:
>>>
>>> 20170418 10:00:41.902 CRITICAL: void CMSConnectionManager::
>>> sltConnectionError(CMSExceptionListener::errorType, QString)
>>> "DataInputStream::readLong - Reached EOF" "tcp://127.0.0.1:61616?
>>> connection.useAsyncSend=true&soKeepAlive=true&wireFormat.
>>> MaxInactivityDuration=240000"
>>>
>>> in the same time on Broker side this is reported in log file:
>>>
>>> 2017-04-18 10:00:29,358 | WARN | Transport Connection to: tcp://
>>> 127.0.0.1:48612 failed: java.io.EOFException |
>> org.apache.activemq.broker.TransportConnection.Transport
>>> | ActiveMQ NIO Worker 8
>>>
>>> questions:
>>>
>>> 1. is there any way to log more detailed information when EOF occurs ? I
>>> mean we don;t want to flood log file with general DEBUG messages but
>> rather
>>> TransportConnection only to figure out what;s going on when EOF appears ?
>>> 2. perhaps we do have something wrong with broker/client settings ?
>>>
>>> broker (part):
>>> <broker xmlns="http://activemq.apache.org/schema/core"
>>> brokerName="localhost" dataDirectory="${activemq.data}" useJmx="true"
>>> advisorySupport="true" persistent="true" schedulePeriodForDestinationPu
>> rge="10000"
>>> networkConnectorStartAsync="true" allowTempAutoCreationOnSend="true"
>>> deleteAllMessagesOnStartup="false" >
>>>
>>> <destinationPolicy>
>>> <policyMap>
>>> <policyEntries>
>>> <policyEntry topic=">" producerFlowControl="false" >
>>> <pendingSubscriberPolicy>
>>> <vmCursor/>
>>> </pendingSubscriberPolicy>
>>> </policyEntry>
>>> <policyEntry queue=">" producerFlowControl="false" >
>>> <pendingQueuePolicy>
>>> <vmQueueCursor/>
>>> </pendingQueuePolicy>
>>> </policyEntry>
>>>
>>> <policyEntry tempTopic="true" producerFlowControl="false" >
>>> <pendingSubscriberPolicy>
>>> <vmCursor/>
>>> </pendingSubscriberPolicy>
>>> </policyEntry>
>>> <policyEntry tempQueue="true" producerFlowControl="false"
>>> gcInactiveDestinations="true" inactiveTimoutBeforeGC="60000">
>>> <pendingQueuePolicy>
>>> <vmQueueCursor/>
>>> </pendingQueuePolicy>
>>> </policyEntry>
>>>
>>> </policyEntries>
>>> </policyMap>
>>> </destinationPolicy>
>>>
>>> <persistenceAdapter>
>>> <levelDB directory="${activemq.data}/leveldb" logSize="107374182"/>
>>> </persistenceAdapter>
>>>
>>>
>>> <systemUsage>
>>> <systemUsage>
>>> <memoryUsage>
>>> <memoryUsage percentOfJvmHeap="80" />
>>> </memoryUsage>
>>> <storeUsage>
>>> <storeUsage limit="20 gb"/>
>>> </storeUsage>
>>> <tempUsage>
>>> <tempUsage limit="10 gb"/>
>>> </tempUsage>
>>> </systemUsage>
>>> </systemUsage>
>>>
>>> <transportConnectors>
>>> <transportConnector name="openwire" uri="nio://0.0.0.0:61616?
>>> transport.maximumConnections=1000&wireFormat.
>>> maxFrameSize=314572800&transport.keepAlive=true&wireFormat.
>>> maxInactivityDuration=240000"/>
>>> </transportConnectors>
>>>
>>> client:
>>> [activemq]
>>> URL="tcp://127.0.0.1:61616?connection.useAsyncSend=true&
>>> soKeepAlive=true&wireFormat.MaxInactivityDuration=240000"
>>>
>>> Thx
>>> Marcin
>>>
>> -
>>
--
Tim Bish
twitter: @tabish121
blog: http://timbish.blogspot.com/
Re: Occasionally EOF on connection
Posted by Tim Bain <tb...@alumni.duke.edu>.
Your right, a connection to localhost should be rock-solid. Sorry I missed
that.
Is there any possibility that a firewall (either OS or application) might
be interfering with your connections? I feel like I'm grasping at straws a
little bit, but I can't explain EOFs on both sides of the connection.
The other variable here is that you're using the CMS client library (which
I have no experience with and which isn't the subject of most posts on this
list), and maybe the EOF it's showing doesn't mean the same thing as an
EOFException would in the Java client library. @Tim Bish, do you see
anything in the described behavior that would be specific to the CMS client
and that might explain what's going on?
Tim
On Apr 20, 2017 8:58 PM, "Marcin Giedz" <ma...@arise.pl> wrote:
> Hi Tim,
>
> the main issue in this case is that this is over localhost connection -
> URL contains 127.0.0.1 ;) we've got plenty of AMQ clients sitting on the
> same host AMQ broker is running.
>
> wireshark is not a problem at all but I wonder how to "narrow" its
> filtering. Perhaps I can also change log4j in AMQ broker to log more
> details on transport connection level but I can't find how to do this
> without global DEBUG/TRACE option ?
>
> Thx
> Marcin
>
>
> Od: "Tim Bain" <tb...@alumni.duke.edu>
> Do: "users" <us...@activemq.apache.org>
> Wysłane: czwartek, 20 kwiecień 2017 7:04:07
> Temat: Re: Occasionally EOF on connection
>
> Nothing in your configuration looks obviously wrong to me.
>
> Is the connection between your broker and the clients that are having
> problems unstable during the period of time that these problems are
> occurring? Simultaneous EOFs on both ends of a socket connection makes me
> think of packet loss or network disconnection; could that be what's going
> on here?
>
> If you're allowed to use it in your environment, Wireshark could help you
> figure out whether the problem is a failure of packets to be sent (e.g. due
> to network problems) or if one side or the other is choosing to close the
> connection.
>
> Tim
>
> On Apr 19, 2017 8:37 AM, "Marcin Giedz" <ma...@arise.pl> wrote:
>
> > Hello all,
> >
> > in our framework we use AMQ 5.14.1 and CMS version 3.9.4. From time to
> > time we observe connection drops on client's side like the one below:
> >
> > 20170418 10:00:41.902 CRITICAL: void CMSConnectionManager::
> > sltConnectionError(CMSExceptionListener::errorType, QString)
> > "DataInputStream::readLong - Reached EOF" "tcp://127.0.0.1:61616?
> > connection.useAsyncSend=true&soKeepAlive=true&wireFormat.
> > MaxInactivityDuration=240000"
> >
> > in the same time on Broker side this is reported in log file:
> >
> > 2017-04-18 10:00:29,358 | WARN | Transport Connection to: tcp://
> > 127.0.0.1:48612 failed: java.io.EOFException |
> org.apache.activemq.broker.TransportConnection.Transport
> > | ActiveMQ NIO Worker 8
> >
> > questions:
> >
> > 1. is there any way to log more detailed information when EOF occurs ? I
> > mean we don;t want to flood log file with general DEBUG messages but
> rather
> > TransportConnection only to figure out what;s going on when EOF appears ?
> > 2. perhaps we do have something wrong with broker/client settings ?
> >
> > broker (part):
> > <broker xmlns="http://activemq.apache.org/schema/core"
> > brokerName="localhost" dataDirectory="${activemq.data}" useJmx="true"
> > advisorySupport="true" persistent="true" schedulePeriodForDestinationPu
> rge="10000"
> > networkConnectorStartAsync="true" allowTempAutoCreationOnSend="true"
> > deleteAllMessagesOnStartup="false" >
> >
> > <destinationPolicy>
> > <policyMap>
> > <policyEntries>
> > <policyEntry topic=">" producerFlowControl="false" >
> > <pendingSubscriberPolicy>
> > <vmCursor/>
> > </pendingSubscriberPolicy>
> > </policyEntry>
> > <policyEntry queue=">" producerFlowControl="false" >
> > <pendingQueuePolicy>
> > <vmQueueCursor/>
> > </pendingQueuePolicy>
> > </policyEntry>
> >
> > <policyEntry tempTopic="true" producerFlowControl="false" >
> > <pendingSubscriberPolicy>
> > <vmCursor/>
> > </pendingSubscriberPolicy>
> > </policyEntry>
> > <policyEntry tempQueue="true" producerFlowControl="false"
> > gcInactiveDestinations="true" inactiveTimoutBeforeGC="60000">
> > <pendingQueuePolicy>
> > <vmQueueCursor/>
> > </pendingQueuePolicy>
> > </policyEntry>
> >
> > </policyEntries>
> > </policyMap>
> > </destinationPolicy>
> >
> > <persistenceAdapter>
> > <levelDB directory="${activemq.data}/leveldb" logSize="107374182"/>
> > </persistenceAdapter>
> >
> >
> > <systemUsage>
> > <systemUsage>
> > <memoryUsage>
> > <memoryUsage percentOfJvmHeap="80" />
> > </memoryUsage>
> > <storeUsage>
> > <storeUsage limit="20 gb"/>
> > </storeUsage>
> > <tempUsage>
> > <tempUsage limit="10 gb"/>
> > </tempUsage>
> > </systemUsage>
> > </systemUsage>
> >
> > <transportConnectors>
> > <transportConnector name="openwire" uri="nio://0.0.0.0:61616?
> > transport.maximumConnections=1000&wireFormat.
> > maxFrameSize=314572800&transport.keepAlive=true&wireFormat.
> > maxInactivityDuration=240000"/>
> > </transportConnectors>
> >
> > client:
> > [activemq]
> > URL="tcp://127.0.0.1:61616?connection.useAsyncSend=true&
> > soKeepAlive=true&wireFormat.MaxInactivityDuration=240000"
> >
> > Thx
> > Marcin
> >
>
> -
>
Re: Occasionally EOF on connection
Posted by Marcin Giedz <ma...@arise.pl>.
Hi Tim,
the main issue in this case is that this is over localhost connection - URL contains 127.0.0.1 ;) we've got plenty of AMQ clients sitting on the same host AMQ broker is running.
wireshark is not a problem at all but I wonder how to "narrow" its filtering. Perhaps I can also change log4j in AMQ broker to log more details on transport connection level but I can't find how to do this without global DEBUG/TRACE option ?
Thx
Marcin
Od: "Tim Bain" <tb...@alumni.duke.edu>
Do: "users" <us...@activemq.apache.org>
Wysłane: czwartek, 20 kwiecień 2017 7:04:07
Temat: Re: Occasionally EOF on connection
Nothing in your configuration looks obviously wrong to me.
Is the connection between your broker and the clients that are having
problems unstable during the period of time that these problems are
occurring? Simultaneous EOFs on both ends of a socket connection makes me
think of packet loss or network disconnection; could that be what's going
on here?
If you're allowed to use it in your environment, Wireshark could help you
figure out whether the problem is a failure of packets to be sent (e.g. due
to network problems) or if one side or the other is choosing to close the
connection.
Tim
On Apr 19, 2017 8:37 AM, "Marcin Giedz" <ma...@arise.pl> wrote:
> Hello all,
>
> in our framework we use AMQ 5.14.1 and CMS version 3.9.4. From time to
> time we observe connection drops on client's side like the one below:
>
> 20170418 10:00:41.902 CRITICAL: void CMSConnectionManager::
> sltConnectionError(CMSExceptionListener::errorType, QString)
> "DataInputStream::readLong - Reached EOF" "tcp://127.0.0.1:61616?
> connection.useAsyncSend=true&soKeepAlive=true&wireFormat.
> MaxInactivityDuration=240000"
>
> in the same time on Broker side this is reported in log file:
>
> 2017-04-18 10:00:29,358 | WARN | Transport Connection to: tcp://
> 127.0.0.1:48612 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport
> | ActiveMQ NIO Worker 8
>
> questions:
>
> 1. is there any way to log more detailed information when EOF occurs ? I
> mean we don;t want to flood log file with general DEBUG messages but rather
> TransportConnection only to figure out what;s going on when EOF appears ?
> 2. perhaps we do have something wrong with broker/client settings ?
>
> broker (part):
> <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="localhost" dataDirectory="${activemq.data}" useJmx="true"
> advisorySupport="true" persistent="true" schedulePeriodForDestinationPurge="10000"
> networkConnectorStartAsync="true" allowTempAutoCreationOnSend="true"
> deleteAllMessagesOnStartup="false" >
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="false" >
> <pendingSubscriberPolicy>
> <vmCursor/>
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="false" >
> <pendingQueuePolicy>
> <vmQueueCursor/>
> </pendingQueuePolicy>
> </policyEntry>
>
> <policyEntry tempTopic="true" producerFlowControl="false" >
> <pendingSubscriberPolicy>
> <vmCursor/>
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry tempQueue="true" producerFlowControl="false"
> gcInactiveDestinations="true" inactiveTimoutBeforeGC="60000">
> <pendingQueuePolicy>
> <vmQueueCursor/>
> </pendingQueuePolicy>
> </policyEntry>
>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> <persistenceAdapter>
> <levelDB directory="${activemq.data}/leveldb" logSize="107374182"/>
> </persistenceAdapter>
>
>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage percentOfJvmHeap="80" />
> </memoryUsage>
> <storeUsage>
> <storeUsage limit="20 gb"/>
> </storeUsage>
> <tempUsage>
> <tempUsage limit="10 gb"/>
> </tempUsage>
> </systemUsage>
> </systemUsage>
>
> <transportConnectors>
> <transportConnector name="openwire" uri="nio://0.0.0.0:61616?
> transport.maximumConnections=1000&wireFormat.
> maxFrameSize=314572800&transport.keepAlive=true&wireFormat.
> maxInactivityDuration=240000"/>
> </transportConnectors>
>
> client:
> [activemq]
> URL="tcp://127.0.0.1:61616?connection.useAsyncSend=true&
> soKeepAlive=true&wireFormat.MaxInactivityDuration=240000"
>
> Thx
> Marcin
>
-
Re: Occasionally EOF on connection
Posted by Tim Bain <tb...@alumni.duke.edu>.
Nothing in your configuration looks obviously wrong to me.
Is the connection between your broker and the clients that are having
problems unstable during the period of time that these problems are
occurring? Simultaneous EOFs on both ends of a socket connection makes me
think of packet loss or network disconnection; could that be what's going
on here?
If you're allowed to use it in your environment, Wireshark could help you
figure out whether the problem is a failure of packets to be sent (e.g. due
to network problems) or if one side or the other is choosing to close the
connection.
Tim
On Apr 19, 2017 8:37 AM, "Marcin Giedz" <ma...@arise.pl> wrote:
> Hello all,
>
> in our framework we use AMQ 5.14.1 and CMS version 3.9.4. From time to
> time we observe connection drops on client's side like the one below:
>
> 20170418 10:00:41.902 CRITICAL: void CMSConnectionManager::
> sltConnectionError(CMSExceptionListener::errorType, QString)
> "DataInputStream::readLong - Reached EOF" "tcp://127.0.0.1:61616?
> connection.useAsyncSend=true&soKeepAlive=true&wireFormat.
> MaxInactivityDuration=240000"
>
> in the same time on Broker side this is reported in log file:
>
> 2017-04-18 10:00:29,358 | WARN | Transport Connection to: tcp://
> 127.0.0.1:48612 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport
> | ActiveMQ NIO Worker 8
>
> questions:
>
> 1. is there any way to log more detailed information when EOF occurs ? I
> mean we don;t want to flood log file with general DEBUG messages but rather
> TransportConnection only to figure out what;s going on when EOF appears ?
> 2. perhaps we do have something wrong with broker/client settings ?
>
> broker (part):
> <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="localhost" dataDirectory="${activemq.data}" useJmx="true"
> advisorySupport="true" persistent="true" schedulePeriodForDestinationPurge="10000"
> networkConnectorStartAsync="true" allowTempAutoCreationOnSend="true"
> deleteAllMessagesOnStartup="false" >
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="false" >
> <pendingSubscriberPolicy>
> <vmCursor/>
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="false" >
> <pendingQueuePolicy>
> <vmQueueCursor/>
> </pendingQueuePolicy>
> </policyEntry>
>
> <policyEntry tempTopic="true" producerFlowControl="false" >
> <pendingSubscriberPolicy>
> <vmCursor/>
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry tempQueue="true" producerFlowControl="false"
> gcInactiveDestinations="true" inactiveTimoutBeforeGC="60000">
> <pendingQueuePolicy>
> <vmQueueCursor/>
> </pendingQueuePolicy>
> </policyEntry>
>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> <persistenceAdapter>
> <levelDB directory="${activemq.data}/leveldb" logSize="107374182"/>
> </persistenceAdapter>
>
>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage percentOfJvmHeap="80" />
> </memoryUsage>
> <storeUsage>
> <storeUsage limit="20 gb"/>
> </storeUsage>
> <tempUsage>
> <tempUsage limit="10 gb"/>
> </tempUsage>
> </systemUsage>
> </systemUsage>
>
> <transportConnectors>
> <transportConnector name="openwire" uri="nio://0.0.0.0:61616?
> transport.maximumConnections=1000&wireFormat.
> maxFrameSize=314572800&transport.keepAlive=true&wireFormat.
> maxInactivityDuration=240000"/>
> </transportConnectors>
>
> client:
> [activemq]
> URL="tcp://127.0.0.1:61616?connection.useAsyncSend=true&
> soKeepAlive=true&wireFormat.MaxInactivityDuration=240000"
>
> Thx
> Marcin
>