You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Jan Rinze <ja...@gmail.com> on 2014/12/12 15:57:35 UTC
Re: heads up: javascript has landed
I have read this thread with great interest.
Will it be possible to use this in a web-page and show incoming messages
from a queue as well as send messages to a queue? We use JSON based web
services to relay between javascript and qpid but sending and receiving
directly to a broker would make life much easier.
Best regards,
Jan Rinze.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Re-heads-up-javascript-has-landed-tp7616982p7617448.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: heads up: javascript has landed
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
On 12/12/14 18:30, Rafael Schloming wrote:
> Hmm, so I hacked websockify to ignore what is specifed in the
> protocols header and hard coded it first to use binary, and second to
> use base64. Neither option seemed to work.
Hi again Rafi,
My bet is that when you hacked Websockify you didn't update the
response, my hack of websockify/websockify.py (do_websocket_handshake)
looks like:
def do_websocket_handshake(self, headers, path):
h = self.headers = headers
self.path = path
prot = 'WebSocket-Protocol'
protocols = h.get('Sec-'+prot, h.get(prot, '')).split(',')
ver = h.get('Sec-WebSocket-Version')
if ver:
# HyBi/IETF version of the protocol
# HyBi-07 report version 7
# HyBi-08 - HyBi-12 report version 8
# HyBi-13 reports version 13
if ver in ['7', '8', '13']:
self.version = "hybi-%02d" % int(ver)
else:
raise self.EClose('Unsupported protocol version %s' % ver)
key = h['Sec-WebSocket-Key']
# Choose binary unless the client explicityly asks for Base64
if 'base64' in protocols:
self.base64 = True
else:
self.base64 = False
#raise self.EClose("Client must support 'binary' or
'base64' protocol")
# Generate the hash value for the accept header
accept = b64encode(sha1(s2b(key + self.GUID)).digest())
response = self.server_handshake_hybi % b2s(accept)
if self.base64:
response += "Sec-WebSocket-Protocol: base64\r\n"
else:
response += "Sec-WebSocket-Protocol: " + ',
'.join(protocols) + "\r\n"
response += "\r\n"
else:
# Hixie version of the protocol (75 or 76)
if h.get('key3'):
trailer = self.gen_md5(h)
pre = "Sec-"
self.version = "hixie-76"
else:
trailer = ""
pre = ""
self.version = "hixie-75"
# We only support base64 in Hixie era
self.base64 = True
response = self.server_handshake_hixie % (pre,
h['Origin'], pre, self.scheme, h['Host'], path)
if 'base64' in protocols:
response += "%sWebSocket-Protocol: base64\r\n" % pre
else:
self.msg("Warning: client does not report 'base64'
protocol support")
response += "\r\n" + trailer
return response
The changed bits are:
# Choose binary unless the client explicitly asks for Base64
if 'base64' in protocols:
self.base64 = True
else:
self.base64 = False
#raise self.EClose("Client must support 'binary' or
'base64' protocol")
and
if self.base64:
response += "Sec-WebSocket-Protocol: base64\r\n"
else:
response += "Sec-WebSocket-Protocol: " + ',
'.join(protocols) + "\r\n"
BTW for info I also tried hacking the JavaScript binding CMakeLists.txt
to remove the
-s \"WEBSOCKET_SUBPROTOCOL='AMQPWSB10'\"
and that works too (emscripten defaults to 'binary' unless a WebSocket
sub-protocol is specified, that's done mainly so Websockify works out of
the box, 'cause Websockify is what's used on the emscripten tests). The
relevant bits are in emscripten/src/settings.js:
// As well as being configurable at compile time via the "-s" option the
WEBSOCKET_URL and WEBSOCKET_SUBPROTOCOL
// settings may configured at run time via the Module object e.g.
// Module['websocket'] = {subprotocol: 'base64, binary, text'};
// Module['websocket'] = {url: 'wss://', subprotocol: 'base64'};
// Run time configuration may be useful as it lets an application select
multiple different services.
var WEBSOCKET_URL = 'ws://'; // A string containing either a WebSocket
URL prefix (ws:// or wss://) or a complete
// RFC 6455 URL - "ws[s]:" "//" host [ ":"
port ] path [ "?" query ].
// In the (default) case of only a prefix
being specified the URL will be constructed from
// prefix + addr + ':' + port
// where addr and port are derived from
the socket connect/bind/accept calls.
var WEBSOCKET_SUBPROTOCOL = 'binary'; // A string containing a comma
separated list of WebSocket subprotocols
// as would be present in the
Sec-WebSocket-Protocol header.
Less drastically than changing this at compile time it's also perfectly
possible to do it at run time.
If you look in say send.js you'll see a line that looks like:
var proton = require("qpid-proton");
if after that line you do:
proton['websocket']['subprotocol'] = 'binary';
That will make it work with an unhacked Websockify too, though as I
mentioned yesterday by setting this to binary rather than AMQPWSB10 it
will work with Websockify but not the Java Broker.
At least the last approach puts control firmly in your hands so you can
make it configurable. Actually I've just this moment tried:
proton['websocket']['subprotocol'] = 'binary, AMQPWSB10';
and that actually works with WebSockify too, but I've not yet checked
whether that would work with the Java Broker, if it does that might be
worth changing the compile/link flag to (though I suspect that it would
make it even more off-spec wrt. the AMQP JavaScript binding spec.).
proxy.js is tolerant to all these shenanigans though :-D I'd be the
first to admit that it needs some work in the error handling department
but it felt worth including 'cause it's pretty simple, is Node.js based
(which is already needed to compile the JavaScript binding, so no extra
dependencies), is Apache licensed.
HTH, and thanks for taking the time to have a play with the JavaScript
binding.
Cheers,
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: heads up: javascript has landed
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Fri, Dec 12, 2014 at 12:39 PM, Fraser Adams <
fraser.adams@blueyonder.co.uk> wrote:
>
>
> On 12/12/14 17:08, Rafael Schloming wrote:
>
>> As an aside, I attempted to use a different web sockets to tcp proxy and
>> I got an error because the protocol wasn't "binary" or "base64". What exact
>> protocol are you using and how is your proxy transcoding to/from web
>> sockets to tcp? --Rafael
>>
> Are you using websockify?
>
Yes, at least I was attempting to use it.
> The proxy isn't doing anything especially clever, the main code is in
> ws2tcp.js. It basically ignores the subprotocol when converting from
> WebSocket to TCP, though it allows you to specify a subprotocol when going
> from TCP to WebSocket. As for "transcoding" it's simply taking the raw
> octets off the WebSocket and putting them onto the TCP socket, i.e. it's a
> transparent proxy.
>
Hmm, so I hacked websockify to ignore what is specifed in the protocols
header and hard coded it first to use binary, and second to use base64.
Neither option seemed to work.
> To answer "what exact protocol are you using" presumably you mean the
> websocket sub-protocol? For that I'm using AMQPWSB10, which is the one
> specified by the AMQP JavaScript Binding. I'm pretty sure that the Java
> Broker WebSocket Transport requires that.
>
> It's also worth mentioning that because I've based the JavaScript stuff
> exactly on top of proton-c I'm literally pushing the same binary octets out
> over the WebSocket as would be pushed over a TCP socket - that's good in
> one sense as it makes proxying to TCP pretty trivial but the AMQP WebSocket
> spec IIRC makes use of the fact that WebSockets are frame based, so uses
> that instead of AMQP framing.
>
The WebSocket spec defines the websockets message payload to be exactly the
same bytes that constitute an AMQP frame, so if you concatenate all the web
sockets messages together as raw binary, you will get a valid sequence of
bytes to send over a TCP connection. The catch is that it goes further to
define an alignment requirement, specifying that every AMQP frame must map
one-to-one onto a websockets message. This complicates things for the tcp
-> ws direction of proxying because the proxy might do partial tcp reads
and end up fragmenting AMQP frames without realizing it.
> The Java Broker is tolerant of being sent full AMQP frames over a
> WebSocket and I personally think that is the best sort of behaviour. Making
> things "fully" compliant with the AMQP WebSocket spec is going to require
> changes to the underlying proton-c and that seemed overkill at this stage.
>
Given the way the spec is defined, what the javascript client is doing is
completely valid (I believe) and should be guaranteed to work with
anything. What the spec messes up is using a transparent proxy, i.e.
essentially forcing you to build/embed a web sockets implementation
directly into your AMQP endpoint (or build a custom proxy that understands
AMQP framing). I think the proper fix for this is to actually relax the
restriction in the proposed spec as it does more harm than good.
> Rob Godfrey mentioned that there was talk of adding an SCTP transport,
> which would also require some refactoring to separate out the framing, but
> IMHO at the moment I quite like it the way it is because *most* things want
> a TCP transport, so being able to proxy fairly transparently is currently a
> really useful thing.
>
> If you look in the CMakeLists.txt in qpid-proton/proton-c/bindings/javascript
> there's a bit that goes:
>
> LINK_FLAGS "-s \"EXPORT_NAME='proton'\" -s \"WEBSOCKET_SUBPROTOCOL='AMQPWSB10'\"
> ${EMSCRIPTEN_LINK_.......
>
>
> I think that if you take out the
> -s \"WEBSOCKET_SUBPROTOCOL='AMQPWSB10'\"
>
> bit it will default to specifying "binary" as the subprotocol, so would
> probably then work with websockify, but probably not then with the Java
> Broker.
>
Thanks, I might play with that if I get some time.
--Rafael
Re: heads up: javascript has landed
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
On 12/12/14 17:08, Rafael Schloming wrote:
> As an aside, I attempted to use a different web sockets to tcp proxy
> and I got an error because the protocol wasn't "binary" or "base64".
> What exact protocol are you using and how is your proxy transcoding
> to/from web sockets to tcp? --Rafael
Are you using websockify?
The proxy isn't doing anything especially clever, the main code is in
ws2tcp.js. It basically ignores the subprotocol when converting from
WebSocket to TCP, though it allows you to specify a subprotocol when
going from TCP to WebSocket. As for "transcoding" it's simply taking the
raw octets off the WebSocket and putting them onto the TCP socket, i.e.
it's a transparent proxy.
To answer "what exact protocol are you using" presumably you mean the
websocket sub-protocol? For that I'm using AMQPWSB10, which is the one
specified by the AMQP JavaScript Binding. I'm pretty sure that the Java
Broker WebSocket Transport requires that.
It's also worth mentioning that because I've based the JavaScript stuff
exactly on top of proton-c I'm literally pushing the same binary octets
out over the WebSocket as would be pushed over a TCP socket - that's
good in one sense as it makes proxying to TCP pretty trivial but the
AMQP WebSocket spec IIRC makes use of the fact that WebSockets are frame
based, so uses that instead of AMQP framing.
The Java Broker is tolerant of being sent full AMQP frames over a
WebSocket and I personally think that is the best sort of behaviour.
Making things "fully" compliant with the AMQP WebSocket spec is going to
require changes to the underlying proton-c and that seemed overkill at
this stage.
Rob Godfrey mentioned that there was talk of adding an SCTP transport,
which would also require some refactoring to separate out the framing,
but IMHO at the moment I quite like it the way it is because *most*
things want a TCP transport, so being able to proxy fairly transparently
is currently a really useful thing.
If you look in the CMakeLists.txt in
qpid-proton/proton-c/bindings/javascript there's a bit that goes:
LINK_FLAGS "-s \"EXPORT_NAME='proton'\" -s
\"WEBSOCKET_SUBPROTOCOL='AMQPWSB10'\" ${EMSCRIPTEN_LINK_.......
I think that if you take out the
-s \"WEBSOCKET_SUBPROTOCOL='AMQPWSB10'\"
bit it will default to specifying "binary" as the subprotocol, so would
probably then work with websockify, but probably not then with the Java
Broker.
Cheers,
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: heads up: javascript has landed
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Fri, Dec 12, 2014 at 12:02 PM, Fraser Adams <
fraser.adams@blueyonder.co.uk> wrote:
>
> Yup pretty much exactly as Rafael says.
>
> If you are using the Qpid Java Broker you need to tweak the config.json in
> qpid-work, basically you need to add something like the following to the
> "ports" array (if you leave out the "id" I think a proper one should get
> created):
>
> {
> "id" : "7f44acca-5de9-4cab-a167-8a1ef7e4119f",
> "authenticationProvider" : "passwordFile",
> "name" : "AMQP-WS",
> "port" : "5673",
> "transports" : [ "WS" ]
> }
>
> Which will activate the WebSocket transport on port 5673
>
> If you are using the C++ broker there is a little Node.js WebSocket->TCP
> Socket proxy provided in:
>
> qpid-proton/examples/messenger/javascript
>
> called simply proxy.js
>
> If you fire that up it will accept a WebSocket connection and connect to a
> TCP socket
>
> ./proxy -h
>
> will show the options.
>
> In that directory there's a little send.html demo but it should be pretty
> easy to do receive too. If you look at recv.js that's set up as a "server"
> ( var address = "amqp://~0.0.0.0";) but if you were to do say:
>
> ./recv.js amqp://0.0.0.0/queue
>
> that would receive messages off a broker - assuming that you had a queue
> called queue and you had enabled the Java Broker WebSocket transport or had
> the proxy running :-)
>
>
> Also (and this *still* bites me :-[ ) be careful of the ports. You
> normally use 5672 for AMQP, but that's the TCP transport, the proxy.js
> defaults to listen on 5673 and sends out on TCP 5672, so you'd generally
> need to do:
>
> ./recv.js amqp://0.0.0.0:5673/queue
>
>
> so basically I've just done:
> 1. open a console and do qpidd --auth no
> 2. do qpid-config add queue queue
> 3. open another console and do (in qpid-proton/examples/messenger/javascript)
> ./proxy.js
> 4. open another console and do (in qpid-proton/examples/messenger/javascript)
> ./recv.js amqp://0.0.0.0:5673/queue
> 5. open another console and do (in qpid-proton/examples/messenger/javascript)
> ./send.js -a amqp://0.0.0.0:5673/queue
>
> It should be fairly easy to hack around with send.html to pull in the
> behaviour of recv.js
>
>
> Be aware that the proxy.js doesn't have much in the way of error handling
> yet if you use the proxy and qpidd and qpidd dies your client won't see it.
> It's on my TODO list, but I've been tied up an a whole bunch of family Xmas
> related things lately, so my weekends of coding have been a bit disrupted
> :-D
It should be *slightly* more robust now. I fixed it so that it wouldn't
fall over when you abort the websockets connection (e.g. from doing a page
reload or something).
As an aside, I attempted to use a different web sockets to tcp proxy and I
got an error because the protocol wasn't "binary" or "base64". What exact
protocol are you using and how is your proxy transcoding to/from web
sockets to tcp?
--Rafael
Re: heads up: javascript has landed
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Yup pretty much exactly as Rafael says.
If you are using the Qpid Java Broker you need to tweak the config.json
in qpid-work, basically you need to add something like the following to
the "ports" array (if you leave out the "id" I think a proper one should
get created):
{
"id" : "7f44acca-5de9-4cab-a167-8a1ef7e4119f",
"authenticationProvider" : "passwordFile",
"name" : "AMQP-WS",
"port" : "5673",
"transports" : [ "WS" ]
}
Which will activate the WebSocket transport on port 5673
If you are using the C++ broker there is a little Node.js WebSocket->TCP
Socket proxy provided in:
qpid-proton/examples/messenger/javascript
called simply proxy.js
If you fire that up it will accept a WebSocket connection and connect to
a TCP socket
./proxy -h
will show the options.
In that directory there's a little send.html demo but it should be
pretty easy to do receive too. If you look at recv.js that's set up as a
"server" ( var address = "amqp://~0.0.0.0";) but if you were to do say:
./recv.js amqp://0.0.0.0/queue
that would receive messages off a broker - assuming that you had a queue
called queue and you had enabled the Java Broker WebSocket transport or
had the proxy running :-)
Also (and this *still* bites me :-[ ) be careful of the ports. You
normally use 5672 for AMQP, but that's the TCP transport, the proxy.js
defaults to listen on 5673 and sends out on TCP 5672, so you'd generally
need to do:
./recv.js amqp://0.0.0.0:5673/queue
so basically I've just done:
1. open a console and do qpidd --auth no
2. do qpid-config add queue queue
3. open another console and do (in
qpid-proton/examples/messenger/javascript) ./proxy.js
4. open another console and do (in
qpid-proton/examples/messenger/javascript) ./recv.js
amqp://0.0.0.0:5673/queue
5. open another console and do (in
qpid-proton/examples/messenger/javascript) ./send.js -a
amqp://0.0.0.0:5673/queue
It should be fairly easy to hack around with send.html to pull in the
behaviour of recv.js
Be aware that the proxy.js doesn't have much in the way of error
handling yet if you use the proxy and qpidd and qpidd dies your client
won't see it. It's on my TODO list, but I've been tied up an a whole
bunch of family Xmas related things lately, so my weekends of coding
have been a bit disrupted :-D
HTH,
Frase
On 12/12/14 15:11, Rafael Schloming wrote:
> Yes, it is possible. The only caveat is that depending on whether your
> broker supports websockets or not you may need to put a websockets to tcp
> proxy in front of your broker.
>
> --Rafael
>
> On Fri, Dec 12, 2014 at 9:57 AM, Jan Rinze <ja...@gmail.com> wrote:
>> I have read this thread with great interest.
>> Will it be possible to use this in a web-page and show incoming messages
>> from a queue as well as send messages to a queue? We use JSON based web
>> services to relay between javascript and qpid but sending and receiving
>> directly to a broker would make life much easier.
>>
>> Best regards,
>> Jan Rinze.
>>
>>
>>
>> --
>> View this message in context:
>> http://qpid.2158936.n2.nabble.com/Re-heads-up-javascript-has-landed-tp7616982p7617448.html
>> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: heads up: javascript has landed
Posted by Rafael Schloming <rh...@alum.mit.edu>.
Yes, it is possible. The only caveat is that depending on whether your
broker supports websockets or not you may need to put a websockets to tcp
proxy in front of your broker.
--Rafael
On Fri, Dec 12, 2014 at 9:57 AM, Jan Rinze <ja...@gmail.com> wrote:
>
> I have read this thread with great interest.
> Will it be possible to use this in a web-page and show incoming messages
> from a queue as well as send messages to a queue? We use JSON based web
> services to relay between javascript and qpid but sending and receiving
> directly to a broker would make life much easier.
>
> Best regards,
> Jan Rinze.
>
>
>
> --
> View this message in context:
> http://qpid.2158936.n2.nabble.com/Re-heads-up-javascript-has-landed-tp7616982p7617448.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>