You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Fraser Adams <fr...@blueyonder.co.uk> on 2015/02/06 14:54:10 UTC
Proton-c Websocket Listeners - thread from off list discussion
Couple of replies inline.
On 06/02/15 13:19, Rafael Schloming wrote:
> A couple of questions/comments inline, but first off, any reason this
> isn't on the list? (I don't mind, just curious.)
No good reason; Alan PMed me and I replied, then though I'd best copy
you guys. It's slightly head explodingly long, so if conversation
develops it might be best to split it up a bit - it was mainly a bit of
a brain dump for Alan and I got carried away :-D I've moved to the Qpid
user and proton lists now for maximum exposure.
>
> On Fri, Feb 6, 2015 at 6:20 AM, Fraser Adams
> <fraser.adams@blueyonder.co.uk <ma...@blueyonder.co.uk>>
> wrote:
>
> Hi Again Alan,
> Sorry this has turned out to be massively TL;DR I kind of got on a
> roll......
>
>
> On the WebSocket thing it's worth pointing out that I've mainly
> been in the game of *using* them and have never had the pleasure
> of writing an implementation from scratch, but OTOH I've done a
> reasonable amount of"hacking around" - but that's mostly been in
> JavaScript and Python implementations so for C things are more
> awkward.
>
> I've copied Rafael so he's in the loop I've also copied Gordon,
> 'cause I'd really quite like WebSocket support in qpidd at some
> point and Rob because he implemented the WebSocket transport in
> the Java Broker and I suspect he knows way more than me -
> especially about the trades around different implementation
> choices, he also knows the AMQP WebSocket spec intricacies way
> better than me so will be better placed to advise gotchas - and
> also I guess an approach that makes is easier for proton-j to
> introduce WebSocket support too.
>
>
> I guess first off I should say that when I suggested it might be a
> car crash if not thought through I was perhaps being a little
> parochial. As you know the JavaScript binding uses emscripten to
> compile proton-c to JavaScript - the way that works is the core
> grammar is compiled to LLVM bitcode and ends up as asm.js but
> library/system calls either wind up calling other compiled code or
> they break out into some pure JavaScript, that's the case for the
> network code and basically all the posix network calls get trapped
> and "faked" in a useful way; library.js and library_sockfs.js are
> the main places to look if you're interested.
>
> I guess that the key point is that all the *socket* calls in
> proton-c wind up as *WebSocket* stuff in the compiled JavaScript,
> for browsers it uses native WebSockets and for Node.js it uses the
> ws WebSocket library, which is I think the most popular one.
>
> The gnarly bit that I can think of is that clearly adding
> WebSocket support to proton-c risks accidentally causing the
> JavaScript stuff to implement WebSocket over WebSocket, which
> would be quite funny but probably not terribly useful :-D
>
>
> It sounds like we quite probably would want to keep any web socket
> stuff compile-time optional and simply not build it at all with
> emscripten. That would a) avoid the possibility of nested websocketry,
> and b) keep useless code out of the already large proton.js file.
It's worth keeping an open mind. The proton-messenger.js stuff it indeed
a bit big, most of that is actually down to the emscripten runtime stuff
and there's a limit to what is easy to strip, there's some work ongoing
on emscripten to make this a bit better. It's <500K minified and it
could be compressed. I'm all for making it smaller but I doubt it'll
make a huge difference I'm afraid. My main concern would be to make sure
that any bits of code to add an API to select between TCP and WebSockets
*weren't* removed I *want* to be able to pass that info to the
emscripten code for when I get round to adding native TCP support.
>
> Another thing I worry a little about is how one implements the
> server side, there are two choices
> 1. Have a separate WebSocket transport
> 2. Note that WebSockets are actually a frame based protocol
> layered on top of TCP and initiated via an HTTP UPGRADE command.
> The spec is here: https://tools.ietf.org/html/rfc6455
>
> The Java Broker (currently) takes the former approach, it's a
> sensible and convenient approach because it uses Jetty as the Web
> Server for its UI and Jetty comes complete with a WebSocket
> implementation, but there's a down side - you have to stand up a
> separate port so basically AMQP over TCP connections connect to
> one port and AMQP over WebSocket connections connect over a
> different port.
>
> But as I say WebSockets are actually layered over TCP so it's
> possible if you have more control over the server to look at the
> data on the TCP buffers and detect that it's actually WebSocket,
> so from a user perspective it would be *really nice* and
> technically possible to have a single listen port.
>
>
> When I suggested implementing it in proton I meant the latter, simply
> autodetecting and upgrading. Andrew has done some nice work around
> this sort of thing already with detecting SSL vs plain and detecting
> SASL vs just core AMQP, so autodetecting the HTTP header and doing the
> upgrade would be a natural and hopefully straightforward way to go. At
> that point you just configure on the transport whether you want to
> allow it or not.
That's cool, I thought that was what you had in mind when you mentioned
Andrew's autodetect stuff but thought I should include Rob and mention
the Java Broker approach to see if there were views on the relative
merits of each.
>
> That's kind of a nice segway into another point
> 1. Use a library
> 2. Roll your own/copy paste reuse
>
> I'm torn on that, using a library is clearly good from a support
> perspective but the counter to that is illustrated by the point I
> make about using Jetty in the Java Broker. Of course you might
> *want* to force TCP and WebSocket connections to different ports
> but do we care more that it's AMQP or what the transport is
> (that's an open question BTW, I personally like the idea of just
> connecting to a single port but there may be good counter arguments).
>
>
> On a slightly library related note it's worth mentioning something
> on my TODO list. As you probably know the JavaScript binding is
> currently limited to using a WebSocket transport, that makes sense
> as a starter for ten 'cause obviously browsers have a number of
> constraints but for Node.js there's no real reason for that to be
> the case. You might have seen the TCP->WebSocket proxy stuff I
> wrote, if you have you'll see that it's actually technically
> pretty easy to implement at TCP version using the Node.js built in
> net library. The main reasons I've not done it yet are:
> 1. Time - I only get to code at weekends and I'm juggling a couple
> of O/S projects at the moment #ifonlyididntneedtoworktolive :-)
> 2. The emscripten network code really isn't as modular as it
> should be - I've done quite a lot of it, but I didn't originate it
> and TBH I'd quite like to pick it apart and make it a bit more
> layered, lack of time again.
> 3. The biggest reason though is actually what approach to take. As
> above I'd really like to have a Node.js AMQP application that can
> just receive either a TCP or a WebSocket connection and *just
> work*. That's a bit awkward because I use ws as the WebSocket
> server. As it's so well established it's a bit daft to roll my own
> but I don't know, it's probably possible to Monkey Patch or
> otherwise expose the native TCP socket if a WebSocket connection
> fails, though probably the best approach would be to add a feature
> to ws and hope that the pull request gets accepted.
>
>
> Hopefully point three above illustrates one potentially awkward
> pinch-point, you'll be adding WebSocket server code in pure C and
> I'll be making the backend of the TCP path all WebSockety too :-)
>
> It *should* work out OK because the emscripten stuff will
> establish a listener and ws will have removed all of the
> WebSockety stuff and just exposes the AMQP frames in the octet
> array that comes off the recv call, so the proton-c code *should*
> just interpret that as AMQP off a TCP socket and not send it to
> any of the proton-c WebSocket code, but depending on how its done
> it could go horribly wrong, which is why I figured I'd give a bit
> of background.
>
>
> All of the above has been from the perspective of a server
> receiving a WebSocket connection, in some respects though
> establishing a connection is actually more awkward....
>
> So it's not like it's technically hard to write the code to have
> Node.js establish an AMQP over TCP connection - the net library is
> pretty straightforward. The bit that I'm actually struggling with
> is how best to let applications choose between a TCP and WebSocket
> transport.
>
> As I say the default position is that the proton-c network code
> maps to emscripten library_sockfs.js calls that push the data over
> WebSockets - so what API to use to tell it to use TCP :-) It's
> easy to set a compile time option but that sucks, I'd like an
> application to be able to connect using either TCP or WebSockets.
>
> I was wondering if the getprotoent /etc/protocols stuff might be
> the best place, that's certainly where you'd get the protocol
> number for UDP, TCP or SCTP to the sockets API but that's used
> pretty much for the underlying Operating System networking and
> WebSockets currently tend to be implemented as user level
> libraries over TCP and I'm not aware of an official protocol entry
> for WebSockets.
>
> That's the only place in the socket API that I can think of that's
> even close to being able to specify that sort of thing though :-/
>
>
> There are certainly other ways to pass that sort of info down to
> emscripten, but as I say I'm struggling for a clean approach.
>
>
> So that conundrum is kind of relevant to the native proton-c
> implementation too 'cause if you want to allow the ability to
> connect using either TCP or WebSockets there needs to be something
> in the API that allows users to choose. Is there anything in any
> of the AMQP specs that have actually covered this? I know that
> Qpid C++ has support for rdma but I *think* that's specified in
> the Connection config which offers a slightly richer set of
> connection options than the sort of amqp connection available in
> Proton (certainly from what I'm aware of with Messenger).
>
>
> I think this should be fairly straightforward in proton-c. We already
> have a similar set of scenarios there with SSL and SASL autodetect.
> When you configure a Transport, you basically construct a stack of
> protocols. In server mode you can it autodetect whether each layer is
> present and fall through to the next. In client mode the stack is just
> what you have specified.
So I can see how the autodetect could be extended to handle this when
acting as a server, but this last bit was really about what sort of
"API" one might use for a client to establish a connection giving a
runtime choice between a TCP or WebSocket transport. As I say for the
qpid::messaging API the Connection allows quite a range of options but
in Messenger at any rate I think the options are more limited and ISTR
there are some restrictions even on SASL options, so as I say it's
probably not "technically" hard I'm just not sure what the best "specify
the transport" API might be for Proton - if you see what I mean?
>
> So I've covered some of the thought processes going through my
> mind as I've been thinking around WebSockets in native proton-c
> and most of them aren't really about the technical implementation
> of the WebSocket protocol, but I think are worth bearing in mind.
>
>
> In terms of actually implementing this stuff as Rafael said I
> think that options in C are fairly limited. As well as the library
> issues I mentioned earlier there's also the little matter of
> worrying about license incompatibilities, so my gut feeling is
> that the best approach might to get "inspiration" from other
> places and to grasp the nettle and implement something that
> integrates nicely with Proton.
>
> If it were me implementing this the first place I'd look would be
> here:
> https://github.com/kanaka/websockify
>
> WebSockify is actually a TCP->WebSocket proxy and TBH bits of it
> irritate me, but itQpid user list now for maximum exposure
> seems to be used quite a lot. The most significant things though
> is that:
> 1. It implements a pretty complete WebSocket stack in Python,
> which you know so should help you figure things out
> 2. It actually also has an implementation in C (under the "other"
> directory) so you might want to look at that
> 3. I *think* that it accepts TCP or WebSocket connections on its
> listen port and just passes through TCP (I can't swear to that but
> it rings a bell)
> 4. It's well used and well maintained so is likely to be a
> reliable starting point
>
> I'd also take a look at ws:
> https://github.com/websockets/ws
>
> This is clearly JavaScript, but it's one of the most commonly used
> and well maintained WebSocket libraries in any language, so again
> as a means to figure out how it actually works it might be a good
> place to look.
>
>
> BTW Rafael mentioned the annoying thing about there being lots of
> different WebSocket versions, that's certainly true, but is much
> less of an issue now. That stuff was generally a pain as the
> WebSocket protocol was evolving, but it has now been standardised
> in the IETF RFC 6455 I linked earlier and all of the main browsers
> primarily support that. Looking on the websockify page there is a
> comment "Starting with websockify 0.5.0, only the HyBi / IETF 6455
> WebSocket protocol is supported". Unless there's a compelling
> customer reason to support pre-standard versions I don't think
> it's unreasonable to only support RFC 6455. It's also worth
> bearing in mind that the emscripten based JavaScript binding
> requires binary support anyway and I think that's only supported
> on the final versions of the spec. so there's probably little
> actual value in supporting anything else.
>
> Sorry for the essay, hope it's useful.
>
> I'm off to focus on my other project now - I'm currently trying to
> write an implementation of webcl using webgl, cause I need to do
> some GPU programming and my platform doesn't have opencl
> support.......
>
> Cheers,
> Frase
>
> On 04/02/15 19:30, Alan Conway wrote:
>
> On Wed, 2015-02-04 at 17:55 +0000, Fraser Adams wrote:
>
> Hi Alan, very quick placeholder response. I've a couple of
> family
> commitments so won't be able to give a decent response for
> a day or so -
> don't want you to think I'm rude by not getting back.
>
> It's a topic that deserves a bit of thought 'cause it
> could be a bit of
> a car-crash if not thought through.
>
> Thanks for the warning! Andrew Stitcher is making changes to the
> transport layer that will affect this (make it easier) so I'm
> not moving
> on proton till that is done. When I do I will consult
> carefully with
> yourself and Andrew to avoid crashing any cars.
>
> I may try some short term hackery to tide me over but if I do
> it will be
> in the dispatch codebase and strictly limited to supporting
> dispatch's
> JS management console. Longer term proton is the right place
> to do it
> and I will avail of all advice so I do it right ;)
>
> Frase.
>
> On 04/02/15 14:46, Alan Conway wrote:
>
> Hi Fraser,
>
> I need a websocket listener in dispatch. Rafi points
> out (rightly) that
> it would be of wider use to do it in proton-c than in
> dispatch. Quick
> JIRA search reveals the word websocket next to your
> name - is that
> Java-only? I know nada about websockets, and not much
> more about JS if
> you remember our prev. conversation ;) Actually on
> that - I now realize
> that (duh!) the java build does the same
> barf-in-the-source-dir thing so
> I was razzing you for nothing. git-ignore is my friend.
>
> Hoping to pick your brains for anything useful on
> websockets. I'm
> guessing my options are
> - find a C websockets library (looks like there are
> some but none
> packaged with popular OS distros (at least the ones
> popular with me)
> - implement the websocket spec in C. Not sure how big
> a job that is yet.
> The spec is full of non-normative waffle so I'm not
> sure just how much
> real action there is.
> - grab a python ws library and just stick with
> dispatch since it has a
> nice C+python implementation. That wouldn't help
> proton though.
>
> Any thoughts?
>
> Cheers
> Alan.
>
>
>
>
Re: Proton-c Websocket Listeners - thread from off list discussion
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Fri, Feb 6, 2015 at 8:54 AM, Fraser Adams <fr...@blueyonder.co.uk>
wrote:
>
>
>> It sounds like we quite probably would want to keep any web socket stuff
>> compile-time optional and simply not build it at all with emscripten. That
>> would a) avoid the possibility of nested websocketry, and b) keep useless
>> code out of the already large proton.js file.
>>
>
> It's worth keeping an open mind. The proton-messenger.js stuff it indeed a
> bit big, most of that is actually down to the emscripten runtime stuff and
> there's a limit to what is easy to strip, there's some work ongoing on
> emscripten to make this a bit better. It's <500K minified and it could be
> compressed. I'm all for making it smaller but I doubt it'll make a huge
> difference I'm afraid. My main concern would be to make sure that any bits
> of code to add an API to select between TCP and WebSockets *weren't*
> removed I *want* to be able to pass that info to the emscripten code for
> when I get round to adding native TCP support.
>
Ah, that's good to know. For some reason I was assuming the overhead was
from our own code, but if it's not then hopefully as you say it will just
improve with time and we don't need to do much about it.
>
>
>> I think this should be fairly straightforward in proton-c. We already
>> have a similar set of scenarios there with SSL and SASL autodetect. When
>> you configure a Transport, you basically construct a stack of protocols. In
>> server mode you can it autodetect whether each layer is present and fall
>> through to the next. In client mode the stack is just what you have
>> specified.
>>
>
> So I can see how the autodetect could be extended to handle this when
> acting as a server, but this last bit was really about what sort of "API"
> one might use for a client to establish a connection giving a runtime
> choice between a TCP or WebSocket transport. As I say for the
> qpid::messaging API the Connection allows quite a range of options but in
> Messenger at any rate I think the options are more limited and ISTR there
> are some restrictions even on SASL options, so as I say it's probably not
> "technically" hard I'm just not sure what the best "specify the transport"
> API might be for Proton - if you see what I mean?
>
>
Yeah, I see what you mean. I suspect there will probably be multiple layers
here. When constructing the transport directly the API choices are fairly
straightforward, but probably overly verbose and flexible for what you
would necessarily want as the common end-user case. I can imagine some sort
of more convenience/configuration oriented layer on top that would let you
just specify all the relevant details in a connect string or something like
that. I believe Gordon already has something like this in the reactive API,
so that might give you an answer.
--Rafael
Re: Proton-c Websocket Listeners - thread from off list discussion
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Fri, Feb 6, 2015 at 8:54 AM, Fraser Adams <fr...@blueyonder.co.uk>
wrote:
>
>
>> It sounds like we quite probably would want to keep any web socket stuff
>> compile-time optional and simply not build it at all with emscripten. That
>> would a) avoid the possibility of nested websocketry, and b) keep useless
>> code out of the already large proton.js file.
>>
>
> It's worth keeping an open mind. The proton-messenger.js stuff it indeed a
> bit big, most of that is actually down to the emscripten runtime stuff and
> there's a limit to what is easy to strip, there's some work ongoing on
> emscripten to make this a bit better. It's <500K minified and it could be
> compressed. I'm all for making it smaller but I doubt it'll make a huge
> difference I'm afraid. My main concern would be to make sure that any bits
> of code to add an API to select between TCP and WebSockets *weren't*
> removed I *want* to be able to pass that info to the emscripten code for
> when I get round to adding native TCP support.
>
Ah, that's good to know. For some reason I was assuming the overhead was
from our own code, but if it's not then hopefully as you say it will just
improve with time and we don't need to do much about it.
>
>
>> I think this should be fairly straightforward in proton-c. We already
>> have a similar set of scenarios there with SSL and SASL autodetect. When
>> you configure a Transport, you basically construct a stack of protocols. In
>> server mode you can it autodetect whether each layer is present and fall
>> through to the next. In client mode the stack is just what you have
>> specified.
>>
>
> So I can see how the autodetect could be extended to handle this when
> acting as a server, but this last bit was really about what sort of "API"
> one might use for a client to establish a connection giving a runtime
> choice between a TCP or WebSocket transport. As I say for the
> qpid::messaging API the Connection allows quite a range of options but in
> Messenger at any rate I think the options are more limited and ISTR there
> are some restrictions even on SASL options, so as I say it's probably not
> "technically" hard I'm just not sure what the best "specify the transport"
> API might be for Proton - if you see what I mean?
>
>
Yeah, I see what you mean. I suspect there will probably be multiple layers
here. When constructing the transport directly the API choices are fairly
straightforward, but probably overly verbose and flexible for what you
would necessarily want as the common end-user case. I can imagine some sort
of more convenience/configuration oriented layer on top that would let you
just specify all the relevant details in a connect string or something like
that. I believe Gordon already has something like this in the reactive API,
so that might give you an answer.
--Rafael