You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by areddy <ar...@better.com> on 2018/10/12 19:43:03 UTC
Connection closed events
I'm using qpid-proton python with activemq.
When i'm sending messages, activemq randomly closes the connection. Here's
an example interaction
Oct 12 14:08:20 chester-2932397692-1zrnb chester-job-worker
[0x559f87f2e4f0]:0 -> @transfer(20) [handle=0, delivery-id=296,
delivery-tag=b"297", message-format=0] (272)
"\x00SpE\x00SsE\x00Sw\xb1\x00\x00\x01\x00{"payload": {"type": "call",
"remote_id": "CA0390195dd4a3591948ae79fb373286be", "data": {"type": "call",
"id": 252832}}, "createdAt": "2018-10-12T18:08:13.705000", "name":
"link_message", "id": "6a98765d-47bf-4f6b-a1ea-548c2877d69d",
"correlationId": null}"
Oct 12 14:08:20 chester-2932397692-1zrnb chester-job-worker
[0x559f87f2e4f0]:0 <- @close(24) [error=@error(29)
[condition=:"amqp:decode-error", description="Could not decode AMQP frame:
hex:
0000013202000000005314d0000000120000000452007000000128a00332393752000053704500537345005377b1000001007b227061796c6f6164223a207b2274797065223a202263616c6c222c202272656d6f74655f6964223a202243413033393031393564643461333539313934386165373966623337333238366265222c202264617461223a207b2274797065223a202263616c6c222c20226964223a203235323833327d7d2c2022637265617465644174223a2022323031382d31302d31325431383a30383a31332e373035303030222c20226e616d65223a20226c696e6b5f6d657373616765222c20226964223a202236613938373635642d343762662d346636622d613165612d353438633238373764363964222c2022636f7272656c6174696f6e4964223a206e756c6c7d"]]
Oct 12 14:08:20 chester-2932397692-1zrnb chester-job-worker ERROR
ERROR:proton:Could not decode AMQP frame: hex:
0000013202000000005314d0000000120000000452007000000128a00332393752000053704500537345005377b1000001007b227061796c6f6164223a207b2274797065223a202263616c6c222c202272656d6f74655f6964223a202243413033393031393564643461333539313934386165373966623337333238366265222c202264617461223a207b2274797065223a202263616c6c222c20226964223a203235323833327d7d2c2022637265617465644174223a2022323031382d31302d31325431383a30383a31332e373035303030222c20226e616d65223a20226c696e6b5f6d657373616765222c20226964223a202236613938373635642d343762662d346636622d613165612d353438633238373764363964222c2022636f7272656c6174696f6e4964223a206e756c6c7d
Oct 12 14:08:20 chester-2932397692-1zrnb chester-job-worker
[0x559f87f2e4f0]:0 -> @close(24) []
Oct 12 14:08:20 chester-2932397692-1zrnb chester-job-worker
[0x559f87f2e4f0]: <- EOS
Oct 12 14:08:20 chester-2932397692-1zrnb chester-job-worker
[0x559f87f2e4f0]: -> EOS
I am not seeing any connection_closed or transport_error events, so i'm
unable to cleanup/reconnect properly.
What events is my MessageHandler supposed to receive when the connection is
closed by remote ?
Why does activemq randomly complain about frames ? Is there something wrong
with the frame ?
--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Connection closed events
Posted by Gordon Sim <gs...@redhat.com>.
On 12/10/18 22:03, Gordon Sim wrote:
> On 12/10/18 21:57, areddy wrote:
>> Thank you. I'll try on_connection_error.
>>
>> This happens once every ~500 messages. Most of the time messages are
>> being
>> transmitted correctly and accepted by activemq. I'm running activemq
>> 5.15.6.
>>
>> *PS:* I'm running a multithreaded application where threads share an
>> instance of the MessageHandler. I added locks to make the send function
>> thread safe.
>
> I'm not sure you can do that safely. It is not just the send but any
> access to the connection. I can't see how you could reliably use locking
> to ensure other threads don't interfere with the event thread. You
> really want to use EventInjector to ensure that only the event thread
> tries to use the connection.
>
> That said the hex looks like it is coming from the broker, and it does
> look like a perfectly valid transfer frame.
(If the problem was that the client was sending badly encoded data due
to some race, I would expect the hex as reported by the broker to be
garbled).
Would be worth running wireshark to see if that spots anything.
Or see if there is any pattern to the failures (are the messages all
essentially of the same form?).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Connection closed events
Posted by Gordon Sim <gs...@redhat.com>.
On 12/10/18 21:57, areddy wrote:
> Thank you. I'll try on_connection_error.
>
> This happens once every ~500 messages. Most of the time messages are being
> transmitted correctly and accepted by activemq. I'm running activemq 5.15.6.
>
> *PS:* I'm running a multithreaded application where threads share an
> instance of the MessageHandler. I added locks to make the send function
> thread safe.
I'm not sure you can do that safely. It is not just the send but any
access to the connection. I can't see how you could reliably use locking
to ensure other threads don't interfere with the event thread. You
really want to use EventInjector to ensure that only the event thread
tries to use the connection.
That said the hex looks like it is coming from the broker, and it does
look like a perfectly valid transfer frame.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Connection closed events
Posted by areddy <ar...@better.com>.
Thank you. I'll try on_connection_error.
This happens once every ~500 messages. Most of the time messages are being
transmitted correctly and accepted by activemq. I'm running activemq 5.15.6.
*PS:* I'm running a multithreaded application where threads share an
instance of the MessageHandler. I added locks to make the send function
thread safe.
--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Connection closed events
Posted by Gordon Sim <gs...@redhat.com>.
On 12/10/18 20:43, areddy wrote:
> Oct 12 14:08:20 chester-2932397692-1zrnb chester-job-worker
> [0x559f87f2e4f0]:0 <- @close(24) [error=@error(29)
> [condition=:"amqp:decode-error", description="Could not decode AMQP frame:
> hex:
> 0000013202000000005314d0000000120000000452007000000128a00332393752000053704500537345005377b1000001007b227061796c6f6164223a207b2274797065223a202263616c6c222c202272656d6f74655f6964223a202243413033393031393564643461333539313934386165373966623337333238366265222c202264617461223a207b2274797065223a202263616c6c222c20226964223a203235323833327d7d2c2022637265617465644174223a2022323031382d31302d31325431383a30383a31332e373035303030222c20226e616d65223a20226c696e6b5f6d657373616765222c20226964223a202236613938373635642d343762662d346636622d613165612d353438633238373764363964222c2022636f7272656c6174696f6e4964223a206e756c6c7d"]]
> Oct 12 14:08:20 chester-2932397692-1zrnb chester-job-worker ERROR
> ERROR:proton:Could not decode AMQP frame: hex:
> 0000013202000000005314d0000000120000000452007000000128a00332393752000053704500537345005377b1000001007b227061796c6f6164223a207b2274797065223a202263616c6c222c202272656d6f74655f6964223a202243413033393031393564643461333539313934386165373966623337333238366265222c202264617461223a207b2274797065223a202263616c6c222c20226964223a203235323833327d7d2c2022637265617465644174223a2022323031382d31302d31325431383a30383a31332e373035303030222c20226e616d65223a20226c696e6b5f6d657373616765222c20226964223a202236613938373635642d343762662d346636622d613165612d353438633238373764363964222c2022636f7272656c6174696f6e4964223a206e756c6c7d
> Oct 12 14:08:20 chester-2932397692-1zrnb chester-job-worker
> [0x559f87f2e4f0]:0 -> @close(24) []
> Oct 12 14:08:20 chester-2932397692-1zrnb chester-job-worker
> [0x559f87f2e4f0]: <- EOS
> Oct 12 14:08:20 chester-2932397692-1zrnb chester-job-worker
> [0x559f87f2e4f0]: -> EOS
>
>
>
> I am not seeing any connection_closed or transport_error events, so i'm
> unable to cleanup/reconnect properly.
>
> What events is my MessageHandler supposed to receive when the connection is
> closed by remote ?
on_connection_error should be called for the close with error
> Why does activemq randomly complain about frames ? Is there something wrong
> with the frame ?
Based on the hex included in the error on close, the frame looks fine to
me. (You could run wireshark to get a second opinion).
You say it is 'randomly' closing the connection. Does that mean that
sometimes similarly structured messages are successfully sent and
accepted by the broker?
It is using the list0 type (i.e. a code for an empty list) for
properties and header. I wonder if that is causing problems on the other
side. (You could try adding a property and e.g. a ttl to it and see if
that alters anything. Of course if other similar messages are succeeding
then that is unlikely to be the issue).
What version of activemq are you using?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org