You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by atani <js...@bepress.com> on 2011/07/20 01:25:48 UTC
Non-ASCII messages via Stomp being dropped or mangled in 5.5
Hi all, I'm running into a problem sending Stomp messages with anything
other than ASCII text in them. I'm connecting from a perl client and I'm
trying to send simple textual messages to a single queue, some of these
messages contain latin1 characters, others unicode characters, others plain
ascii. No matter whether I explicitly encode the message bodies to unicode
or not, ActiveMQ appears to be collapsing multiple messages into a single
message when it receives a message with non-ASCII. An example is probably
the easiest way to show this and I apologize for the verbosity.
Attached to this message is a perl script that sends this set of messages,
first without performing any explicit encoding and then after encoding the
body as utf-8. Both cases see the error when a frame contains non-ascii -
it seems to glob later frames together.
Does anybody have any idea of what might be going on here? This is really
strange.
The output of the script shows the Net::Stomp::Frame's as they are sent to
the server. here:
% perl -I stomplib/ stomptest3.pl
:FRAME_START:
CONNECT
passcode:blahblahblah
login:system
:END_OF_FRAME:
:FRAME_START:
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 not encoded: I'm a simple message
:END_OF_FRAME:
:FRAME_START:
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 not encoded: I'm another simple message
:END_OF_FRAME:
:FRAME_START:
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 not encoded: latin1 here>ØoÖps☻< smiles
:END_OF_FRAME:
:FRAME_START:
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 utf8 encoded: I'm a simple message
:END_OF_FRAME:
:FRAME_START:
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 utf8 encoded: I'm another simple message
:END_OF_FRAME:
:FRAME_START:
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 utf8 encoded: latin1 here>ÃoÃpsâ»< smiles
:END_OF_FRAME:
:FRAME_START:
DISCONNECT
:END_OF_FRAME:
And the ActiveMQ server log has the following corresponding entries:
2011-07-19 16:18:12,552 [2.5.9.197:60260] TRACE StompTransportFilter
- Received:
CONNECT
passcode:*****
login:system
2011-07-19 16:18:12,554 [2.5.9.197:60260] TRACE StompTransportFilter
- Sending:
CONNECTED
session:ID:mq.util.bepress.com-45374-1311095072884-4:412
2011-07-19 16:18:13,784 [2.5.9.197:60260] TRACE StompTransportFilter
- Received:
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 not encoded: I'm a simple message
2011-07-19 16:18:13,785 [2.5.9.197:60260] TRACE StompTransportFilter
- Received:
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 not encoded: I'm ano...mple message
2011-07-19 16:18:13,785 [2.5.9.197:60260] TRACE StompTransportFilter
- Received:
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 not encoded: latin1 ...mple message
2011-07-19 16:18:13,785 [2.5.9.197:60260] TRACE StompTransportFilter
- Received:
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 utf8 encoded: I'm an...mple message
2011-07-19 16:18:13,785 [2.5.9.197:60260] TRACE StompTransportFilter
- Received:
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 utf8 encoded: latin1...DISCONNECT
The actual messages received show that the third through the fifth message
get crunched into a single message for some reason. The third message body
has latin1 characters in it, the fourth has unicode, and the fifth is back
to ascii. It's almost as if the first appearance of the non-ascii causes
incoming messages to become appended to the non-ascii message until a
pure-ascii message again arrives.
Message 1 body # # # # # # # # # # # # # # # # # # # # # # # # #
Tue Jul 19 16:18:13 2011 not encoded: I'm a simple message
Message 2 body # # # # # # # # # # # # # # # # # # # # # # # # #
Tue Jul 19 16:18:13 2011 not encoded: I'm another simple message
Message 3 body # # # # # # # # # # # # # # # # # # # # # # # # #
This one is three messages in one!
Tue Jul 19 16:18:13 2011 not encoded: latin1 here>ØoÖps?< smileSEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 16:18:13 2011 utf8 encoded: I'm a simple message
Message 4 body # # # # # # # # # # # # # # # # # # # # # # # # #
Tue Jul 19 16:18:13 2011 utf8 encoded: I'm another simple message
Message 5 body # # # # # # # # # # # # # # # # # # # # # # # # #
This one is two messages in one, with the ending DISCONNECT!
Tue Jul 19 16:18:13 2011 utf8 encoded: latin1 here>ØoÖps☻<
smilDISCONNECT
Further, using the nc utility I can verify that the stomp SEND Frames are
being sent across the wire correctly:
$ nc -l -p 61616 | perl -ne '$|=1; $_ =~ s/\0/[NUL]/g; print $_;'
CONNECT
passcode:blahblahblah
login:system
CONNECTED
session-id: 1234
^@
[NUL]SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 15:50:44 2011 not encoded: I'm a simple message[NUL]SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 15:50:44 2011 not encoded: I'm another simple message[NUL]SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 15:50:44 2011 not encoded: latin1 here>?o?ps<here[NUL]SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 15:50:44 2011 utf8 encoded: I'm a simple message[NUL]SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 15:50:44 2011 utf8 encoded: I'm another simple message[NUL]SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 15:50:44 2011 utf8 encoded: latin1 here>ØoÖps<here[NUL]SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 15:50:44 2011 utf8 encoded: unicode >☻< smiles[NUL]SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 15:50:44 2011 utf8 encoded: I'm a simple message[NUL]SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 15:50:44 2011 utf8 encoded: I'm another simple message[NUL]SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 15:50:44 2011 utf8 encoded: latin1 here>ØoÖps<here[NUL]SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
Tue Jul 19 15:50:44 2011 utf8 encoded: unicode >☻< smiles[NUL]DISCONNECT
[NUL]
There is no "not encoded" version of the unicode body because the sending
side correctly never sends that one due to the fact that it is not encoded
and a "wide character in syswrite" error occurs when trying to send it.
Thanks for any help you can provide. I've not been able to find anything in
the archives about this particular issue.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Non-ASCII-messages-via-Stomp-being-dropped-or-mangled-in-5-5-tp3679601p3679601.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Non-ASCII messages via Stomp being dropped or mangled in 5.5
Posted by Dejan Bosanac <de...@nighttale.net>.
Thanks for closing the loop.
Regards
--
Dejan Bosanac - http://twitter.com/dejanb
-----------------
The experts in open source integration and messaging - http://fusesource.com
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Thu, Jul 28, 2011 at 2:03 AM, atani <js...@bepress.com> wrote:
>
> I downloaded the 5.6 source and added trace logging in StompWireFormat.java
> and LegacyFrameTranslator.java and it's starting to look like there's
> something happening between the Net::Stomp library and the ActiveMQ input
> stream - that something is somehow related to the use of SSL. The problem
> only occurs when I'm connected over SSL (possibly something in
> IO::Socket::SSL).
>
>
> Without SSL my StompWireFormat logging shows a fine looking byte[]
> constructed in unmarshal( DataInput in ) and everything works well:
>
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - - - - - - - - unmarshal
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - parseHeaders line: destination:/queue/jstevenson@jstevenson2.msgtest2
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - parseHeaders line: content-length:63
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - parseHeaders line:
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - header 'destination' = '/queue/jstevenson@jstevenson2.msgtest2'
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - header 'content-length' = '63'
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - data 0 W / 87
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - data 1 e / 101
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - data 2 d / 100
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - data 3 / 32
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - data 4 J / 74
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - data 5 u / 117
> 2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
> - data 6 l / 108
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 7 / 32
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 8 2 / 50
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 9 7 / 55
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 10 / 32
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 11 1 / 49
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 12 6 / 54
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 13 : / 58
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 14 1 / 49
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 15 2 / 50
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 16 : / 58
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 17 5 / 53
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 18 1 / 49
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 19 / 32
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 20 2 / 50
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 21 0 / 48
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 22 1 / 49
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 23 1 / 49
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 24 / 32
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 25 u / 117
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 26 t / 116
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 27 f / 102
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 28 8 / 56
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 29 / 32
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 30 e / 101
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 31 n / 110
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 32 c / 99
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 33 o / 111
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 34 d / 100
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 35 e / 101
> 2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
> - data 36 d / 100
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 37 : / 58
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 38 / 32
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 39 l / 108
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 40 a / 97
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 41 t / 116
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 42 i / 105
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 43 n / 110
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 44 1 / 49
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 45 / 32
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 46 h / 104
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 47 e / 101
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 48 r / 114
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 49 e / 101
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 50 > / 62
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 51 ? / -61
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 52 ? / -104
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 53 o / 111
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 54 ? / -61
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 55 ? / -106
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 56 p / 112
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 57 s / 115
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 58 < / 60
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 59 h / 104
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 60 e / 101
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 61 r / 114
> 2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
> - data 62 e / 101
> 2011-07-27 16:12:51,643 [.168.1.61:46444] TRACE StompWireFormat
> - - - - - - - - done/unmarshal
>
>
> Over SSL that logging shows that the byte[] has additional bytes in it, at
> the point of the encoded non-ascii characters.
>
>
> 2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
> - - - - - - - - unmarshal
> 2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
> - parseHeaders line: destination:/queue/jstevenson@jstevenson2.msgtest2
> 2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
> - parseHeaders line: content-length:63
> 2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
> - parseHeaders line:
> 2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
> - header 'destination' = '/queue/jstevenson@jstevenson2.msgtest2'
> 2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
> - header 'content-length' = '63'
> 2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
> - data 0 W / 87
> 2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
> - data 1 e / 101
> 2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
> - data 2 d / 100
> 2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
> - data 3 / 32
> 2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
> - data 4 J / 74
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 5 u / 117
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 6 l / 108
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 7 / 32
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 8 2 / 50
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 9 7 / 55
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 10 / 32
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 11 1 / 49
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 12 6 / 54
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 13 : / 58
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 14 0 / 48
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 15 3 / 51
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 16 : / 58
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 17 1 / 49
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 18 5 / 53
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 19 / 32
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 20 2 / 50
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 21 0 / 48
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 22 1 / 49
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 23 1 / 49
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 24 / 32
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 25 u / 117
> 2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
> - data 26 t / 116
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 27 f / 102
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 28 8 / 56
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 29 / 32
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 30 e / 101
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 31 n / 110
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 32 c / 99
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 33 o / 111
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 34 d / 100
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 35 e / 101
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 36 d / 100
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 37 : / 58
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 38 / 32
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 39 l / 108
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 40 a / 97
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 41 t / 116
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 42 i / 105
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 43 n / 110
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 44 1 / 49
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 45 / 32
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 46 h / 104
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 47 e / 101
> 2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
> - data 48 r / 114
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 49 e / 101
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 50 > / 62
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 51 ￃ / -61
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 52 ﾃ / -125
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 53 ￂ / -62
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 54 ﾘ / -104
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 55 o / 111
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 56 ￃ / -61
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 57 ﾃ / -125
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 58 ￂ / -62
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 59 ﾖ / -106
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 60 p / 112
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 61 s / 115
> 2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
> - data 62 < / 60
>
>
> The problems start at array index 51. So this looks like it's either
> something in the SSL encoding layer (perl) or the decoding layer - though I
> don't think it's the decoding/receiving end: I've tried the ActiveMQ
> stomp+ssl transportConnector, stunnel, and other SSL offloading utilities,
> the behavior is the same.
>
>
> Thanks all for their thoughts and input - perhaps I'll try Lionel's
> Net::Stomp::Client but I might just call this enough and go with the base64
> approach, as unfortunate as will be.
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Non-ASCII-messages-via-Stomp-being-dropped-or-mangled-in-5-5-tp3679601p3700108.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Non-ASCII messages via Stomp being dropped or mangled in 5.5
Posted by Aleksandar Ivanisevic <al...@ivanisevic.de>.
atani <js...@bepress.com> writes:
> I downloaded the 5.6 source and added trace logging in StompWireFormat.java
> and LegacyFrameTranslator.java and it's starting to look like there's
> something happening between the Net::Stomp library and the ActiveMQ input
> stream - that something is somehow related to the use of SSL. The problem
> only occurs when I'm connected over SSL (possibly something in
> IO::Socket::SSL).
What version of Net::Stomp are you using? I have noticed something
similar with the latest version (0.4x) on CPAN (messages were
sometimes chopped at 65535 bytes) so I went back to 0.35 which has
been working flawlessly for me for a year or so.
[...]
Re: Non-ASCII messages via Stomp being dropped or mangled in 5.5
Posted by atani <js...@bepress.com>.
I downloaded the 5.6 source and added trace logging in StompWireFormat.java
and LegacyFrameTranslator.java and it's starting to look like there's
something happening between the Net::Stomp library and the ActiveMQ input
stream - that something is somehow related to the use of SSL. The problem
only occurs when I'm connected over SSL (possibly something in
IO::Socket::SSL).
Without SSL my StompWireFormat logging shows a fine looking byte[]
constructed in unmarshal( DataInput in ) and everything works well:
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- - - - - - - - unmarshal
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- parseHeaders line: destination:/queue/jstevenson@jstevenson2.msgtest2
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- parseHeaders line: content-length:63
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- parseHeaders line:
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- header 'destination' = '/queue/jstevenson@jstevenson2.msgtest2'
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- header 'content-length' = '63'
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- data 0 W / 87
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- data 1 e / 101
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- data 2 d / 100
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- data 3 / 32
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- data 4 J / 74
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- data 5 u / 117
2011-07-27 16:12:51,640 [.168.1.61:46444] TRACE StompWireFormat
- data 6 l / 108
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 7 / 32
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 8 2 / 50
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 9 7 / 55
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 10 / 32
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 11 1 / 49
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 12 6 / 54
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 13 : / 58
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 14 1 / 49
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 15 2 / 50
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 16 : / 58
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 17 5 / 53
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 18 1 / 49
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 19 / 32
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 20 2 / 50
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 21 0 / 48
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 22 1 / 49
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 23 1 / 49
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 24 / 32
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 25 u / 117
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 26 t / 116
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 27 f / 102
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 28 8 / 56
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 29 / 32
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 30 e / 101
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 31 n / 110
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 32 c / 99
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 33 o / 111
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 34 d / 100
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 35 e / 101
2011-07-27 16:12:51,641 [.168.1.61:46444] TRACE StompWireFormat
- data 36 d / 100
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 37 : / 58
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 38 / 32
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 39 l / 108
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 40 a / 97
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 41 t / 116
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 42 i / 105
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 43 n / 110
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 44 1 / 49
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 45 / 32
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 46 h / 104
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 47 e / 101
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 48 r / 114
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 49 e / 101
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 50 > / 62
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 51 ? / -61
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 52 ? / -104
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 53 o / 111
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 54 ? / -61
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 55 ? / -106
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 56 p / 112
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 57 s / 115
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 58 < / 60
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 59 h / 104
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 60 e / 101
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 61 r / 114
2011-07-27 16:12:51,642 [.168.1.61:46444] TRACE StompWireFormat
- data 62 e / 101
2011-07-27 16:12:51,643 [.168.1.61:46444] TRACE StompWireFormat
- - - - - - - - done/unmarshal
Over SSL that logging shows that the byte[] has additional bytes in it, at
the point of the encoded non-ascii characters.
2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
- - - - - - - - unmarshal
2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
- parseHeaders line: destination:/queue/jstevenson@jstevenson2.msgtest2
2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
- parseHeaders line: content-length:63
2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
- parseHeaders line:
2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
- header 'destination' = '/queue/jstevenson@jstevenson2.msgtest2'
2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
- header 'content-length' = '63'
2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
- data 0 W / 87
2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
- data 1 e / 101
2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
- data 2 d / 100
2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
- data 3 / 32
2011-07-27 16:03:15,594 [127.0.0.1:38819] TRACE StompWireFormat
- data 4 J / 74
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 5 u / 117
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 6 l / 108
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 7 / 32
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 8 2 / 50
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 9 7 / 55
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 10 / 32
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 11 1 / 49
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 12 6 / 54
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 13 : / 58
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 14 0 / 48
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 15 3 / 51
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 16 : / 58
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 17 1 / 49
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 18 5 / 53
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 19 / 32
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 20 2 / 50
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 21 0 / 48
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 22 1 / 49
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 23 1 / 49
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 24 / 32
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 25 u / 117
2011-07-27 16:03:15,595 [127.0.0.1:38819] TRACE StompWireFormat
- data 26 t / 116
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 27 f / 102
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 28 8 / 56
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 29 / 32
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 30 e / 101
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 31 n / 110
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 32 c / 99
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 33 o / 111
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 34 d / 100
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 35 e / 101
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 36 d / 100
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 37 : / 58
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 38 / 32
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 39 l / 108
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 40 a / 97
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 41 t / 116
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 42 i / 105
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 43 n / 110
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 44 1 / 49
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 45 / 32
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 46 h / 104
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 47 e / 101
2011-07-27 16:03:15,596 [127.0.0.1:38819] TRACE StompWireFormat
- data 48 r / 114
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 49 e / 101
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 50 > / 62
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 51 ￃ / -61
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 52 ﾃ / -125
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 53 ￂ / -62
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 54 ﾘ / -104
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 55 o / 111
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 56 ￃ / -61
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 57 ﾃ / -125
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 58 ￂ / -62
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 59 ﾖ / -106
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 60 p / 112
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 61 s / 115
2011-07-27 16:03:15,597 [127.0.0.1:38819] TRACE StompWireFormat
- data 62 < / 60
The problems start at array index 51. So this looks like it's either
something in the SSL encoding layer (perl) or the decoding layer - though I
don't think it's the decoding/receiving end: I've tried the ActiveMQ
stomp+ssl transportConnector, stunnel, and other SSL offloading utilities,
the behavior is the same.
Thanks all for their thoughts and input - perhaps I'll try Lionel's
Net::Stomp::Client but I might just call this enough and go with the base64
approach, as unfortunate as will be.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Non-ASCII-messages-via-Stomp-being-dropped-or-mangled-in-5-5-tp3679601p3700108.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Non-ASCII messages via Stomp being dropped or mangled in 5.5
Posted by Dejan Bosanac <de...@nighttale.net>.
Hi,
you can always set some traces on
https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-core/src/main/java/org/apache/activemq/transport/stomp/LegacyFrameTranslator.java?hb=true
and see what's going on. Or create a test case similar to
https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-core/src/test/java/org/apache/activemq/transport/stomp/StompTest.java?hb=true#to424
so we can inspect it further
Regards
--
Dejan Bosanac - http://twitter.com/dejanb
-----------------
The experts in open source integration and messaging - http://fusesource.com
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Thu, Jul 21, 2011 at 9:59 PM, atani <js...@bepress.com> wrote:
>
> Hi Dejan,
>
>
> UTF-8 encoded characters can be 1 to 3 bytes in length; characters less
> than
> or equal to 127 (hex 0x7F) is one byte, 128 to 2048 (0x07FF) is two bytes,
> 2049 to 65535 (0xFFFF) is three bytes. e.g. the smiley character in my
> test
> encodes to three bytes (0xE2, 0x98, and 0xBB) and the Ö encodes to two
> (0xC3
> and 0x96). From what I can tell the stomp messages contain the correct
> bytes in the body when I'm sending them with a content-length header.
>
>
> The Stomp message contains the correct bytes in the body when they're being
> sent with a content-length header. As text, the utf8 message is:
>
> Wed Jul 20 10:32:50 2011 utf8 encoded: unicode >☻< smiles - 57 characters
> long
>
>
> but in the stomp "bytes message" it is:
>
> Wed Jul 20 10:32:50 2011 utf8 encoded: unicode >☻< smiles - 59 bytes
> long,
> which is the length reported in the content-length header.
>
>
>
> In the Stomp protocol I'm not sure what you mean by "convert it explicitly
> to a byte array". The Stomp message being sent with the content-length
> header contains the bytes that result from encoding the message text in
> utf-8 and the content-length header reports that number of bytes
> accurately.
> Unfortunately I can't get the full message / actual bytes received logged
> in
> the ActiveMQ stomp log - is there another class that I could crank the
> logging up on to get the incoming message logged in full?
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Non-ASCII-messages-via-Stomp-being-dropped-or-mangled-in-5-5-tp3679601p3684826.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Non-ASCII messages via Stomp being dropped or mangled in 5.5
Posted by atani <js...@bepress.com>.
Hi Dejan,
UTF-8 encoded characters can be 1 to 3 bytes in length; characters less than
or equal to 127 (hex 0x7F) is one byte, 128 to 2048 (0x07FF) is two bytes,
2049 to 65535 (0xFFFF) is three bytes. e.g. the smiley character in my test
encodes to three bytes (0xE2, 0x98, and 0xBB) and the Ö encodes to two (0xC3
and 0x96). From what I can tell the stomp messages contain the correct
bytes in the body when I'm sending them with a content-length header.
The Stomp message contains the correct bytes in the body when they're being
sent with a content-length header. As text, the utf8 message is:
Wed Jul 20 10:32:50 2011 utf8 encoded: unicode >☻< smiles - 57 characters
long
but in the stomp "bytes message" it is:
Wed Jul 20 10:32:50 2011 utf8 encoded: unicode >â»< smiles - 59 bytes long,
which is the length reported in the content-length header.
In the Stomp protocol I'm not sure what you mean by "convert it explicitly
to a byte array". The Stomp message being sent with the content-length
header contains the bytes that result from encoding the message text in
utf-8 and the content-length header reports that number of bytes accurately.
Unfortunately I can't get the full message / actual bytes received logged in
the ActiveMQ stomp log - is there another class that I could crank the
logging up on to get the incoming message logged in full?
--
View this message in context: http://activemq.2283324.n4.nabble.com/Non-ASCII-messages-via-Stomp-being-dropped-or-mangled-in-5-5-tp3679601p3684826.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Non-ASCII messages via Stomp being dropped or mangled in 5.5
Posted by Dejan Bosanac <de...@nighttale.net>.
As far as I know unicode chars are two-bytes long, so that could be a cause.
You should convert it explicitly to byte array and then send it.
Regards
--
Dejan Bosanac - http://twitter.com/dejanb
-----------------
The experts in open source integration and messaging - http://fusesource.com
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Wed, Jul 20, 2011 at 8:08 PM, atani <js...@bepress.com> wrote:
>
> Thanks, Dejan. Fiddling with sending a "BytesMessage" results in another
> interesting error.
> When I send a plain ASCII-containing bytes message specifying a
> content-length equal
> to the length of the message body not including the terminating null byte,
> everything
> is fine. But when I send a message that contains the byte encoded form of
> a
> unicode
> string, and a content-length equal to the length of the body in bytes but
> not
> including the terminating null byte, an exception is thrown.
>
>
> : org.apache.activemq.transport.stomp.ProtocolException: content-length
> bytes were read and there was no trailing null byte
> 2011-07-20 10:32:50,715 [2.5.9.197:56330] DEBUG ProtocolConverter
> - Exception detail
> org.apache.activemq.transport.stomp.ProtocolException: content-length bytes
> were read and there was no trailing null byte
> at
>
> org.apache.activemq.transport.stomp.StompWireFormat.unmarshal(StompWireFormat.java:109)
> at
>
> org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:227)
> at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219)
> at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202)
> at java.lang.Thread.run(Thread.java:662)
>
>
> Frame 1 - OK
>
> ASCII characters sent as bytes with a content-length header equal to the
> length
> of the *body*. The terminating null byte is replaced in my log with the
> [NUL]
> seen below so I can confirm it's presence - but the frame is sent with the
> actual
> null byte.
>
>
> SEND
> destination:/queue/jstevenson@jstevenson2.msgtest2
> content-length:59
>
> Wed Jul 20 10:32:50 2011 utf8 encoded: I'm a simple message[NUL]
>
>
> Frame 2 - Not OK
>
> Unicode characters encoded into bytes with a content-length header also
> equal to the
> length of the *body*. A similar failure occurs with encoded latin1
> characters.
>
>
> SEND
> destination:/queue/jstevenson@jstevenson2.msgtest2
> content-length:59
>
> Wed Jul 20 10:32:50 2011 utf8 encoded: unicode >☻< smiles[NUL]
>
>
>
> I will ping the Net::Stomp maintainer about this as well, but it does seem
> like
> there might be a problem in the receiving Stomp protocol handler. The docs
> about
> the use of the content-length header being used as a trigger for handling
> the message
> as either a BytesMessage or a TextMessage are a little unclear to me:
>
> It is recommended that SEND frames include a content-length header which is
> > a byte count for the length of the message body. If a content-length
> > header is
> > included, this number of bytes should be read, regardless of whether or
> > not
> > there are null characters in the body. The frame still needs to be
> > terminated
> > with a null byte and if a content-length is not specified, the first null
> > byte
> > encountered signals the end of the frame.
>
> (emphasis mine around "The frame"). This looks like it's saying that the
> content-length should indicate the length of the body not including the
> frame's terminating null byte while the frame itself is still terminated by
> the
> null byte which is not included in the content-length header figure. This
> does
> appear to be what the Net::Stomp library is doing; but am I missing
> something?
>
>
> Thanks again for any clarification and help.
>
>
> -Joel
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Non-ASCII-messages-via-Stomp-being-dropped-or-mangled-in-5-5-tp3679601p3681683.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Non-ASCII messages via Stomp being dropped or mangled in 5.5
Posted by atani <js...@bepress.com>.
Thanks, Dejan. Fiddling with sending a "BytesMessage" results in another
interesting error.
When I send a plain ASCII-containing bytes message specifying a
content-length equal
to the length of the message body not including the terminating null byte,
everything
is fine. But when I send a message that contains the byte encoded form of a
unicode
string, and a content-length equal to the length of the body in bytes but
not
including the terminating null byte, an exception is thrown.
: org.apache.activemq.transport.stomp.ProtocolException: content-length
bytes were read and there was no trailing null byte
2011-07-20 10:32:50,715 [2.5.9.197:56330] DEBUG ProtocolConverter
- Exception detail
org.apache.activemq.transport.stomp.ProtocolException: content-length bytes
were read and there was no trailing null byte
at
org.apache.activemq.transport.stomp.StompWireFormat.unmarshal(StompWireFormat.java:109)
at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:227)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202)
at java.lang.Thread.run(Thread.java:662)
Frame 1 - OK
ASCII characters sent as bytes with a content-length header equal to the
length
of the *body*. The terminating null byte is replaced in my log with the
[NUL]
seen below so I can confirm it's presence - but the frame is sent with the
actual
null byte.
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
content-length:59
Wed Jul 20 10:32:50 2011 utf8 encoded: I'm a simple message[NUL]
Frame 2 - Not OK
Unicode characters encoded into bytes with a content-length header also
equal to the
length of the *body*. A similar failure occurs with encoded latin1
characters.
SEND
destination:/queue/jstevenson@jstevenson2.msgtest2
content-length:59
Wed Jul 20 10:32:50 2011 utf8 encoded: unicode >â»< smiles[NUL]
I will ping the Net::Stomp maintainer about this as well, but it does seem
like
there might be a problem in the receiving Stomp protocol handler. The docs
about
the use of the content-length header being used as a trigger for handling
the message
as either a BytesMessage or a TextMessage are a little unclear to me:
It is recommended that SEND frames include a content-length header which is
> a byte count for the length of the message body. If a content-length
> header is
> included, this number of bytes should be read, regardless of whether or
> not
> there are null characters in the body. The frame still needs to be
> terminated
> with a null byte and if a content-length is not specified, the first null
> byte
> encountered signals the end of the frame.
(emphasis mine around "The frame"). This looks like it's saying that the
content-length should indicate the length of the body not including the
frame's terminating null byte while the frame itself is still terminated by
the
null byte which is not included in the content-length header figure. This
does
appear to be what the Net::Stomp library is doing; but am I missing
something?
Thanks again for any clarification and help.
-Joel
--
View this message in context: http://activemq.2283324.n4.nabble.com/Non-ASCII-messages-via-Stomp-being-dropped-or-mangled-in-5-5-tp3679601p3681683.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Non-ASCII messages via Stomp being dropped or mangled in 5.5
Posted by Dejan Bosanac <de...@nighttale.net>.
Hi,
Stomp protocol supports simple text messages and binary (byte) messages. See
in your Perl library how to send binary messages and convert your payload to
byte arrays and it should work (or use base64 encoding to convert it to
text).
Regards
--
Dejan Bosanac - http://twitter.com/dejanb
-----------------
The experts in open source integration and messaging - http://fusesource.com
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Wed, Jul 20, 2011 at 1:28 AM, atani <js...@bepress.com> wrote:
> Oops, somehow lost the ability to attach the script to the previous post,
> so
> here it is.
> http://activemq.2283324.n4.nabble.com/file/n3679606/stomptest3.pl
> stomptest3.pl
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Non-ASCII-messages-via-Stomp-being-dropped-or-mangled-in-5-5-tp3679601p3679606.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Non-ASCII messages via Stomp being dropped or mangled in 5.5
Posted by atani <js...@bepress.com>.
Oops, somehow lost the ability to attach the script to the previous post, so
here it is.
http://activemq.2283324.n4.nabble.com/file/n3679606/stomptest3.pl
stomptest3.pl
--
View this message in context: http://activemq.2283324.n4.nabble.com/Non-ASCII-messages-via-Stomp-being-dropped-or-mangled-in-5-5-tp3679601p3679606.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.