You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Rajesh Malla <ma...@gmail.com> on 2018/03/09 01:20:14 UTC
Camel generating message with correlationid=null and tmp-queue null
Hi
we are using mesh n/w connectors A,B,C. and we have two consumers. upto some
hours system is working fine, after that when we are checking messages those
are generated by camel with correlationId as null
ActiveMQObjectMessage {commandId = 4, responseRequired = true, messageId =
ID:asdf-55403-1520511160238-20:1:1:1:1, originalDestination = null,
originalTransactionId = null, producerId =
ID:asdf-55403-1520511160238-20:1:1:1, destination = queue://uniquequeuename,
transactionId = null, expiration = 0, timestamp = 1520552897962, arrival =
0, brokerInTime = 1520552897962, brokerOutTime = 1520553651034,
correlationId = null, replyTo = null, persistent = true, type = null,
priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null,
compressed = false, userID = admin, content =
org.apache.activemq.util.ByteSequence@6fa51cd4, marshalledProperties =
org.apache.activemq.util.ByteSequence@2767e23c, dataStructure = null,
redeliveryCounter = 0, size = 1607, properties =
{CamelAuthentication_Bytes=rO0ABXQAD2FkbWluCUFETUlOCUFMTA==,
breadcrumbId=ID-asdf-55404-1520511167186-2-14}, readOnlyProperties = true,
readOnlyBody = true, droppable = false, jmsXGroupFirstForConsumer = false}
and replyTo also become null. because of this, consumer is receiving [ from
test client ] and not able to respond to that message.
upto some hours it is working properly, but this is happening after few
hours. Is this the problem with Camel ? activemq configuration please
suggest.
original message :
ActiveMQObjectMessage {commandId = 40, responseRequired = true, messageId =
ID:asdf-58894-1520512195342-12:1:10:1:57, originalDestination = null,
originalTransactionId = null, producerId =
new-broker->new-broker-58894-1520512195342-20:2:1:1, destination =
queue://uniquequeuename, transactionId = null, expiration = 1520555745459,
timestamp = 1520555685459, arrival = 0, brokerInTime = 1520555685462,
brokerOutTime = 1520555734467, correlationId =
Camel-ID-asdf-41356-1520512199669-2-203, replyTo =
temp-queue://ID:asdf-58894-1520512195342-12:1:2, persistent = true, type =
null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId =
null, compressed = false, userID = admin, content =
org.apache.activemq.util.ByteSequence@cc43f62, marshalledProperties =
org.apache.activemq.util.ByteSequence@5b218417, dataStructure = null,
redeliveryCounter = 0, size = 1488, properties =
{CamelAuthentication_Bytes=rO0ABXQAD2FkbWluCUFETUlOCUFMTA==,
breadcrumbId=ID-asdf-41356-1520512199669-2-201}, readOnlyProperties = true,
readOnlyBody = true, droppable = false, jmsXGroupFirstForConsumer = false}
Please help us.
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel generating message with correlationid=null and tmp-queue
null
Posted by Rajesh Malla <ma...@gmail.com>.
it seems blockingQueue is not actually ACTiveMQDestination, just it is java,
so any other suggestions ?
at that state we are not able to connect to cloud broker , on client side we
are getting :
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(Unknown Source)[:1.8.0_60]
at
org.apache.activemq.transport.util.TextWireFormat.unmarshal(TextWireFormat.java:52)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.transport.http.HttpClientTransport.run(HttpClientTransport.java:196)[120:org.apache.activemq.activemq-osgi:5.12.3]
... 1 more
also, we are getting this only when we are adding networkpeer. we also tried
removing of InactivityMonitor and also tried increasing
setConnectAttemptTimeout to 5 minutes. but still issue exists.
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel generating message with correlationid=null and tmp-queue null
Posted by Tim Bain <tb...@alumni.duke.edu>.
Rajesh,
First, I want to let you know that although it's possible to edit/delete
mailing list posts on the Nabble site after you've posted them, the edits
are not sent out to the mailing list via email, so anyone (like me) who
works primarily from email will not see the changes you make. So the best
workflow is the only one that will work for both modes: treat posts as
uneditable once they're sent initially, and make any corrections or
additions to them by creating follow-on posts. If you choose to edit/delete
posts once they're initially created, it will be harder for me (and anyone
else using a workflow based on email) to support you.
With that out of the way, I'm struggling to understand exactly what your
question is. Clearly a portion of your question is why your clients are
being disconnected from your broker, and I'll come to that in the next
paragraph, but is that the only question? You started out asking about why
certain properties on your messages were null, but you've not asked about
that in several posts; is that issue resolved? You then asked about why
your pooled connections were failing due to "channel has already failed;"
is that issue still relevant? You also asked (in posts that you've since
removed from the Nabble site) something about blockingQueue and its
relationship to dynamic destination inclusion in a network of brokers; does
the fact that you've removed that content from the Nabble site mean that it
also is no longer a question you need answers to?
For the question of why your connections are being disconnected, the
InactivityIOException that you added to the Nabble post on 3/24 seems to
indicate that the connection is being terminated because it's been inactive
for longer than the inactivity timeout interval allows. Do you have the
inactivity monitor enabled? If so, what settings are you using, on both
sides of the connection? Details about the inactivity monitor and how it's
configured can be found at
http://activemq.apache.org/activemq-inactivitymonitor.html.
Have you confirmed that the firewall between your client and your broker is
not filtering out any traffic between these two processes. If you haven't
confirmed that every byte is making it through in both directions, I'd
suggest you make that a priority.
BTW, there was no file named jstack.out containing a thread dump attached
to your message either in email or in the Nabble post. If it's still
relevant to your question, can you please re-send that?
Tim
On Sat, Mar 24, 2018 at 4:12 AM, Rajesh Malla <ma...@gmail.com> wrote:
> Thank you Tim
>
> for this IOException, we have not observed any reasonable exceptions on
> broker [ producer ] side.
>
> attached thread dump jstack.out. Can you please look into and help.
>
>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>
Re: Camel generating message with correlationid=null and tmp-queue
null
Posted by Rajesh Malla <ma...@gmail.com>.
Thank you Tim
for this IOException, we have not observed any reasonable exceptions on
broker [ producer ] side.
attached thread dump jstack.out. Can you please look into and help.
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel generating message with correlationid=null and tmp-queue
null
Posted by Rajesh Malla <ma...@gmail.com>.
thank you Tim,
>>Caused by: java.io.IOException: Failed to perform GET on: https://URL
>>Reason: null
>>>>>>at
org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:33)[120:org.apache.activemq.activemq-osgi:5.12.3]
>>>>>>at
org.apache.activemq.transport.http.HttpClientTransport.run(HttpClientTransport.java:205)[120:org.apache.activemq.activemq-osgi:5.12.3]
>>>>>>... 1 more
>>Caused by: java.io.EOFException
>>>>>>at java.io.DataInputStream.readInt(Unknown Source)[:1.8.0_60]
>>>>>>at
org.apache.activemq.transport.util.TextWireFormat.unmarshal(TextWireFormat.java:52)[120:org.apache.activemq.activemq-osgi:5.12.3]
>>>>>>at
org.apache.activemq.transport.http.HttpClientTransport.run(HttpClientTransport.java:196)[120:org.apache.activemq.activemq-osgi:5.12.3]
>>>>>>... 1 more
>>Is this the root cause of why your connections are already closed when
>>trying to open sessions, etc.? If so, there's one of two possibilities:
>>1. The broker is experiencing errors that cause it to close the connection
>>prematurely. What do you see in the broker logs? Everything you've
provided
>>so far has been from the consumer's logs.
>>2. The load balancer is experiencing behavior that cause it to terminate
>>the connection prematurely, e.g. due to inactivity or other problems. If
>>there's nothing in the broker's logs that indicates a problem, this would
>>be the next place to dig into.
for the IOException, we are seeing below problem
clientID: ID:7450-51820-1521801292483-0:1
2018-03-23 06:35:01,302 58688438 [CustHttpTransportServer_1926295252-120]
(org.apache.activemq.activemq-osgi:org.apache.activemq.transport.http.HttpTunnelServlet:175)
WARN - The clientID header specified is invalid. Client sesion has not yet
been established for it: ID:7450-51820-1521801292483-0:1
so, we are suspecting this is coming bcz blockingqueus are not able to
create, in our n/w peer we removed
networkConnector.addDynamicallyIncludedDestination condition because of this
even blocking queues are also replicated which is causing this problem
[assumption].
so blocking queues are created with *blockingQueue_<SOME NUMBER>*
can you please suggest what pattern to exclude this blocking queue. or any
other suggestions ?
if we exclude blockingQueue_* or blockingQueue.* ?
or any other way we need to follow ? please suggest
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel generating message with correlationid=null and tmp-queue null
Posted by Tim Bain <tb...@alumni.duke.edu>.
On Sun, Mar 18, 2018 at 9:14 AM, Rajesh Malla <ma...@gmail.com> wrote:
> We have to use https only, because our server broker sits in cloud &
> onpremises in another location. via some other url authenticating. this
> scenario is working properly in <5.8.0
> of activemq however not working with 5.12.3 version. I mean, for some
> hours it is working after that it is giving errors like attached :
> error.txt <http://activemq.2283324.n4.nabble.com/file/t378915/error.txt>
>
OK, I won't push further on this. The concerns I've voiced about the
limitations of the HTTP transport wouldn't cause these connections to fail,
which implies that there's something else going on.
The following content is in one of the stack traces you provided in that
output:
Caused by: java.io.IOException: Failed to perform GET on: https://URL
Reason: null
at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:33)[120:org.apache.activemq.activemq-osgi:5.12.3]
at org.apache.activemq.transport.http.HttpClientTransport.run(HttpClientTransport.java:205)[120:org.apache.activemq.activemq-osgi:5.12.3]
... 1 more
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(Unknown Source)[:1.8.0_60]
at org.apache.activemq.transport.util.TextWireFormat.unmarshal(TextWireFormat.java:52)[120:org.apache.activemq.activemq-osgi:5.12.3]
at org.apache.activemq.transport.http.HttpClientTransport.run(HttpClientTransport.java:196)[120:org.apache.activemq.activemq-osgi:5.12.3]
... 1 more
Is this the root cause of why your connections are already closed when
trying to open sessions, etc.? If so, there's one of two possibilities:
1. The broker is experiencing errors that cause it to close the connection
prematurely. What do you see in the broker logs? Everything you've provided
so far has been from the consumer's logs.
2. The load balancer is experiencing behavior that cause it to terminate
the connection prematurely, e.g. due to inactivity or other problems. If
there's nothing in the broker's logs that indicates a problem, this would
be the next place to dig into.
Otherwise, I'd attach a debugger to the client and put a breakpoint in
org.apache.activemq.jms.pool.PooledSession.close() and try to figure out
who's calling it and why.
> we are using service mix [ activemq 5.12.3 & camel 2.16.2 & spring 3.2 ].
>
> one more question : when we are making route using camel we are seeing many
> consumers are started.
> for spring DMLC we are providing default cachel level CACHE_NONE. for one
> route why it is creating nearly 8 to 9 consumers on same queue. because of
> above errors each consumer is closed I think, finally nothing is working.
>
You'd need to give more information about your usage pattern for anyone to
conclusively answer this question. One possibility is that the repeated
failure to connect creates 8-9 consumers in succession, with all of them in
a failed state. Another is that you're creating 8-9 DMLCs, either
intentionally or due to retries from exception handling. Or maybe it's
something else; I'm not an expert on the behavior of the DMLC, so if you
feel that this question is relevant to the broader questions in this
thread, you might want to pose it on a Spring forum or on StackOverflow to
see if someone who knows the DMLC code well can shed some light on why it's
doing what it's doing.
> https option we cannot change it now, bcz of our designs, what is the main
> reason we are not able to receive message ?
>
> also remove connection and create connection means, we are removing
> component in camel and creating new route in camel on that queue. even if
> we
> restart client also wroking.
BTW, have you read
http://activemq.2283324.n4.nabble.com/Logentry-quot-Setup-of-JMS-message-listener-invoker-failed-for-destination-quot-td4676507.html
and
http://activemq.2283324.n4.nabble.com/ActiveMQ-Pooled-Session-td4698203.html?
Both provide proposed solutions from other users, though in neither case
did the OP respond to say whether the suggested changes had the desired
effect, so I can't promise that they'll work for you (nor that they won't
cause some problem). But if either one does work, we'd certainly like to
know that.
Tim
Re: Camel generating message with correlationid=null and tmp-queue
null
Posted by Rajesh Malla <ma...@gmail.com>.
>Is your client using a failover transport? If so, it's the failover
>transport that's re-establishing the connection.
we are not using any failover transport
I see you're using the https transport. Does it work if you connect with
the tcp transport instead? It's possible that there's some bug in the https
transport, so if you find that clients can connect with tcp but not with
https, that would be useful information to know.. ??
We have to use https only, because our server broker sits in cloud &
onpremises in another location. via some other url authenticating. this
scenario is working properly in <5.8.0
of activemq however not working with 5.12.3 version. I mean, for some
hours it is working after that it is giving errors like attached :
error.txt <http://activemq.2283324.n4.nabble.com/file/t378915/error.txt>
we are using service mix [ activemq 5.12.3 & camel 2.16.2 & spring 3.2 ].
one more question : when we are making route using camel we are seeing many
consumers are started.
for spring DMLC we are providing default cachel level CACHE_NONE. for one
route why it is creating nearly 8 to 9 consumers on same queue. because of
above errors each consumer is closed I think, finally nothing is working.
https option we cannot change it now, bcz of our designs, what is the main
reason we are not able to receive message ?
also remove connection and create connection means, we are removing
component in camel and creating new route in camel on that queue. even if we
restart client also wroking.
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel generating message with correlationid=null and tmp-queue null
Posted by Tim Bain <tb...@alumni.duke.edu>.
On Tue, Mar 13, 2018 at 6:52 PM, Rajesh Malla <ma...@gmail.com> wrote:
> when we are looking into Server broker, it has many temperary queues around
> more than 2000. So we have removed one network peer.
> A->B
> B->A
> now, after some hours tmp-queues are around 2 digits only. we have applied
> below patch :
> https://issues.apache.org/jira/browse/AMQ-6262
>
> still after some hours client is not able to re-establish connection. I
> have
> questions like below
>
> 1) InactivityMonitor each time close connection, if this is the case who is
> again re-establishing connection ?
>
Is your client using a failover transport? If so, it's the failover
transport that's re-establishing the connection.
> 2) after some hours our client is not able to re-establish connection. on
> client side we are seeing different errors
>
> 2018-03-13 18:12:24,091 386450649 [ActiveMQ Connection Executor: HTTP
> Reader
> https:URL] ( CustomisedPooledConnectionFactory:114) WARN - Connection
> error occurred for https:URL: Cannot send, channel has already failed:
> https:URL
>
>
> (org.apache.servicemix.bundles.spring-jms:org.
> springframework.jms.listener.DefaultMessageListenerContainer:909)
> ERROR - Could not refresh JMS Connection for destination 'queue' -
> retrying
> in 20000 ms. Cause: The JMS connection has failed: Channel was inactive (no
> connection attempt made) for too (>60000) long: https://URL
>
> 2018-03-13 13:25:16,292 369222850 [Camel (camel-4) thread #378 -
> TemporaryQueueReplyManager[ ]]
> (org.apache.servicemix.bundles.spring-jms:org.
> springframework.jms.listener.DefaultMessageListenerContainer:909)
> ERROR - Could not refresh JMS Connection for destination 'temporary' -
> retrying in 5000 ms. Cause: The JMS connection has failed: Channel was
> inactive (no connection attempt made) for too (>60000) long: https://URL
>
I see you're using the https transport. Does it work if you connect with
the tcp transport instead? It's possible that there's some bug in the https
transport, so if you find that clients can connect with tcp but not with
https, that would be useful information to know..
> from client side when we are again manually removing and setting connection
>
Please define the terms "removing connection" and "setting connection,"
because those phrases don't mean anything to me.
> then it is working for sometime and after that it is not working.
>
> our scenario is simple A->B n/w mesh with one client. Please suggest any,
>
I'm never going to recommend that you use the http/https transports, so...
My suggestion is that you ditch them and use the tcp transport (i.e. the
OpenWire protocol). HTTP without websockets is a sub-optimal way to consume
JMS messages because it interacts poorly with non-durable topic subscribers
(it'll only establish a subscription for as long as the connection is
open), so I'd recommend you switch to tcp. The http/https transports are,
to my mind, only useful when you're working in a non-Java environment
(which clearly you're not, since you're using Camel), so I don't see a
reason you should be using them.
> let us know if we want to provide any other information.
>
Re: Camel generating message with correlationid=null and tmp-queue
null
Posted by Rajesh Malla <ma...@gmail.com>.
when we are looking into Server broker, it has many temperary queues around
more than 2000. So we have removed one network peer.
A->B
B->A
now, after some hours tmp-queues are around 2 digits only. we have applied
below patch :
https://issues.apache.org/jira/browse/AMQ-6262
still after some hours client is not able to re-establish connection. I have
questions like below
1) InactivityMonitor each time close connection, if this is the case who is
again re-establishing connection ?
2) after some hours our client is not able to re-establish connection. on
client side we are seeing different errors
2018-03-13 18:12:24,091 386450649 [ActiveMQ Connection Executor: HTTP Reader
https:URL] ( CustomisedPooledConnectionFactory:114) WARN - Connection
error occurred for https:URL: Cannot send, channel has already failed:
https:URL
(org.apache.servicemix.bundles.spring-jms:org.springframework.jms.listener.DefaultMessageListenerContainer:909)
ERROR - Could not refresh JMS Connection for destination 'queue' - retrying
in 20000 ms. Cause: The JMS connection has failed: Channel was inactive (no
connection attempt made) for too (>60000) long: https://URL
2018-03-13 13:25:16,292 369222850 [Camel (camel-4) thread #378 -
TemporaryQueueReplyManager[ ]]
(org.apache.servicemix.bundles.spring-jms:org.springframework.jms.listener.DefaultMessageListenerContainer:909)
ERROR - Could not refresh JMS Connection for destination 'temporary' -
retrying in 5000 ms. Cause: The JMS connection has failed: Channel was
inactive (no connection attempt made) for too (>60000) long: https://URL
from client side when we are again manually removing and setting connection
then it is working for sometime and after that it is not working.
our scenario is simple A->B n/w mesh with one client. Please suggest any,
let us know if we want to provide any other information.
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel generating message with correlationid=null and tmp-queue null
Posted by Tim Bain <tb...@alumni.duke.edu>.
Was that error happening when you first saw the behavior you first asked
about? Are you saying that these are related, or that they're two unrelated
issues? I can't see anything obvious about this exception that I would
expect to result in the behavior you saw in your first post, so my starting
assumption is that the two are unrelated.
Tim
On Thu, Mar 8, 2018 at 9:58 PM, Rajesh Malla <ma...@gmail.com> wrote:
> following error messages we are seeing on client side :
> 2018-03-08 18:38:24,248 41319597 [Camel (jcsCoreCamelContext) thread #12 -
> JmsConsumer[uniquequename]]
> (org.apache.activemq.activemq-osgi:org.apache.activemq.jms.
> pool.PooledSession:126)
> WARN - Caught exception trying close() when putting session back into the
> pool, will invalidate. javax.jms.JMSException: Cannot send, channel has
> already failed: https://URL
> javax.jms.JMSException: Cannot send, channel has already failed:
> https://URL
> at
> org.apache.activemq.util.JMSExceptionSupport.create(
> JMSExceptionSupport.java:72)[120:org.apache.activemq.activemq-osgi:5.12.3]
> at
> org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(
> ActiveMQConnection.java:1314)[120:org.apache.activemq.
> activemq-osgi:5.12.3]
> at
> org.apache.activemq.ActiveMQConnection.asyncSendPacket(
> ActiveMQConnection.java:1306)[120:org.apache.activemq.
> activemq-osgi:5.12.3]
> at
> org.apache.activemq.ActiveMQSession.asyncSendPacket(
> ActiveMQSession.java:2074)[120:org.apache.activemq.activemq-osgi:5.12.3]
> at
> org.apache.activemq.ActiveMQMessageConsumer.doClose(
> ActiveMQMessageConsumer.java:728)[120:org.apache.activemq.
> activemq-osgi:5.12.3]
> at
> org.apache.activemq.ActiveMQMessageConsumer.close(
> ActiveMQMessageConsumer.java:715)[120:org.apache.activemq.
> activemq-osgi:5.12.3]
> at
> org.apache.activemq.jms.pool.PooledSession.close(
> PooledSession.java:108)[120:org.apache.activemq.activemq-osgi:5.12.3]
> at
> org.apache.activemq.jms.pool.PooledConnection.cleanupAllLoanedSessions(
> PooledConnection.java:257)[120:org.apache.activemq.activemq-osgi:5.12.3]
> at
> org.apache.activemq.jms.pool.PooledConnection.close(
> PooledConnection.java:85)[120:org.apache.activemq.activemq-osgi:5.12.3]
> at
> org.springframework.jms.connection.ConnectionFactoryUtils.
> releaseConnection(ConnectionFactoryUtils.java:
> 80)[140:org.apache.servicemix.bundles.spring-jms:3.2.14.RELEASE_1]
> at
> org.springframework.jms.listener.AbstractJmsListeningContainer.
> refreshSharedConnection(AbstractJmsListeningContainer.
> java:387)[140:org.apache.servicemix.bundles.spring-jms:3.2.14.RELEASE_1]
> at
> org.springframework.jms.listener.DefaultMessageListenerContainer.
> refreshConnectionUntilSuccessful(DefaultMessageListenerContaine
> r.java:887)[140:org.apache.servicemix.bundles.spring-jms:3.2.14.RELEASE_1]
> at
> org.springframework.jms.listener.DefaultMessageListenerContainer.
> recoverAfterListenerSetupFailure(DefaultMessageListenerContaine
> r.java:863)[140:org.apache.servicemix.bundles.spring-jms:3.2.14.RELEASE_1]
> at
> org.springframework.jms.listener.DefaultMessageListenerContaine
> r$AsyncMessageListenerInvoker.run(DefaultMessageListenerContaine
> r.java:1014)[140:org.apache.servicemix.bundles.spring-jms:
> 3.2.14.RELEASE_1]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)[:1.8.0_60]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)[:1.8.0_60]
> at java.lang.Thread.run(Unknown Source)[:1.8.0_60]
> Caused by: org.apache.activemq.transport.InactivityIOException: Cannot
> send,
> channel has already failed: https://URL
> at
> org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(
> AbstractInactivityMonitor.java:333)[120:org.apache.
> activemq.activemq-osgi:5.12.3]
> at
> org.apache.activemq.transport.AbstractInactivityMonitor.oneway(
> AbstractInactivityMonitor.java:322)[120:org.apache.
> activemq.activemq-osgi:5.12.3]
> at
> org.apache.activemq.transport.MutexTransport.oneway(
> MutexTransport.java:68)[120:org.apache.activemq.activemq-osgi:5.12.3]
> at
> org.apache.activemq.transport.ResponseCorrelator.oneway(
> ResponseCorrelator.java:60)[120:org.apache.activemq.activemq-osgi:5.12.3]
> at
> org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(
> ActiveMQConnection.java:1312)[120:org.apache.activemq.
> activemq-osgi:5.12.3]
> ... 15 more
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>
Re: Camel generating message with correlationid=null and tmp-queue
null
Posted by Rajesh Malla <ma...@gmail.com>.
following error messages we are seeing on client side :
2018-03-08 18:38:24,248 41319597 [Camel (jcsCoreCamelContext) thread #12 -
JmsConsumer[uniquequename]]
(org.apache.activemq.activemq-osgi:org.apache.activemq.jms.pool.PooledSession:126)
WARN - Caught exception trying close() when putting session back into the
pool, will invalidate. javax.jms.JMSException: Cannot send, channel has
already failed: https://URL
javax.jms.JMSException: Cannot send, channel has already failed: https://URL
at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:72)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1314)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1306)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:2074)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.ActiveMQMessageConsumer.doClose(ActiveMQMessageConsumer.java:728)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.ActiveMQMessageConsumer.close(ActiveMQMessageConsumer.java:715)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.jms.pool.PooledSession.close(PooledSession.java:108)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.jms.pool.PooledConnection.cleanupAllLoanedSessions(PooledConnection.java:257)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.jms.pool.PooledConnection.close(PooledConnection.java:85)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.springframework.jms.connection.ConnectionFactoryUtils.releaseConnection(ConnectionFactoryUtils.java:80)[140:org.apache.servicemix.bundles.spring-jms:3.2.14.RELEASE_1]
at
org.springframework.jms.listener.AbstractJmsListeningContainer.refreshSharedConnection(AbstractJmsListeningContainer.java:387)[140:org.apache.servicemix.bundles.spring-jms:3.2.14.RELEASE_1]
at
org.springframework.jms.listener.DefaultMessageListenerContainer.refreshConnectionUntilSuccessful(DefaultMessageListenerContainer.java:887)[140:org.apache.servicemix.bundles.spring-jms:3.2.14.RELEASE_1]
at
org.springframework.jms.listener.DefaultMessageListenerContainer.recoverAfterListenerSetupFailure(DefaultMessageListenerContainer.java:863)[140:org.apache.servicemix.bundles.spring-jms:3.2.14.RELEASE_1]
at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1014)[140:org.apache.servicemix.bundles.spring-jms:3.2.14.RELEASE_1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source)[:1.8.0_60]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)[:1.8.0_60]
at java.lang.Thread.run(Unknown Source)[:1.8.0_60]
Caused by: org.apache.activemq.transport.InactivityIOException: Cannot send,
channel has already failed: https://URL
at
org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:333)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.transport.AbstractInactivityMonitor.oneway(AbstractInactivityMonitor.java:322)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)[120:org.apache.activemq.activemq-osgi:5.12.3]
at
org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1312)[120:org.apache.activemq.activemq-osgi:5.12.3]
... 15 more
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Camel generating message with correlationid=null and tmp-queue null
Posted by Tim Bain <tb...@alumni.duke.edu>.
I'll start with the bad news. From what you've written, it could be either
product (Camel or ActiveMQ) that's causing the problem, and the people that
support the two products are both likely to initially assume it's the other
product's doing. So one of the best things you can do is to figure out
which product the problem is occurring in, so you can get the right people
involved in troubleshooting what's going on.
To that end, one thing you could do is try cycling the ActiveMQ Broker and
seeing whether the problem continues. Then wait for the problem to happen
again, and cycle the Camel producer and see whether the problem continues.
If cycling one product stops the problem from happening for a while and
cycling the other has no effect, then the problem is almost certainly in
the one that's fixed by cycling. If the behavior is the same no matter
which you cycle, then you'll have to look for another way to determine
where the problem lies.
I'd also suggest turning on TRACE logging in both products to see if you
can see a log dump of each message that's sent by Camel and received by the
broker. If you can find a log line that's close to the sending of the
message, and it shows that the message is correct at the time it crosses
the wire to the broker, then the problem is likely in ActiveMQ; if it's
already incorrect by the time it goes across the wire, then it's likely a
Camel problem.
Can you reproduce this problem reliably (even if it takes a few hours)? Or
was this a one-time fluke that you've never seen happen again?
Tim
P.S. I also notice that the expiration is 0 in the message that's not
right, whereas it's got a non-zero value for the correct message.
On Thu, Mar 8, 2018 at 9:11 PM, Rajesh Malla <ma...@gmail.com> wrote:
>
> we are using activemq : 5.12.3
> apache camel : 2.16.2
>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>
Re: Camel generating message with correlationid=null and tmp-queue
null
Posted by Rajesh Malla <ma...@gmail.com>.
we are using activemq : 5.12.3
apache camel : 2.16.2
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html