You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by Bruce Mitchener <br...@gmail.com> on 2010/04/08 00:13:34 UTC
Thoughts on an RPC protocol
I'm assuming that the goals of an optimized transport for Avro RPC are
something like the following:
* Framing should be efficient, easy to implement.
* Streaming of large values, both as part of a request and as a response is
very important.
* Being able to have multiple concurrent requests in flight, while also
being able to have ordering guarantees where desired is necessary.
* It should be easy to implement this in Java, C, Python, Ruby, etc.
* Security is or will be important. This security can include authorization
as well as privacy concerns.
I'd like to see something based largely upon RFC 3080, with some
simplifications and extensions:
http://www.faqs.org/rfcs/rfc3080.html
What does this get us?
* This system has mechanisms in place for streaming both a single large
message and breaking a single reply up into multiple answers, allowing for
pretty flexible streaming. (You can even mix these by having an answer that
gets chunked itself.)
* Concurrency is achieved by having multiple channels. Each channel
executes messages in order, so you have a good mechanism for sending
multiple things at once as well as maintaining ordering guarantees as
necessary.
* Reporting errors is very clear as it is a separate response type.
* It has already been specified pretty clearly and we'd just be evolving
that to something that more closely matches our needs.
* It specifies sufficient data that you could implement this over
transports other than TCP, such as UDP.
Changes, rough list:
* Use Avro-encoding for most things, so the encoding of a message would
become an Avro struct.
* Lose profiles in the sense that they're used in that specification since
we're just exchanging Avro RPCs.
* Do length prefixing rather than in the header, so that it is very
amenable to binary I/O at high volumes.
* No XML stuff, just existing things like the Avro handshake, wrapped up in
messages.
* For now, don't worry about things like flow control as expressed in RFC
3081, mapping of 3080 to TCP.
* Think about adding something for true one-way messages, but an empty
reply frame is probably sufficient, since that still allows reporting errors
if needed (or desired).
* May well need some extensions for a more flexible security model.
* Use Avro RPC stuff to encode the channel management commands on channel 0
rather than XML.
RFC 3117 (http://www.faqs.org/rfcs/rfc3117.html) goes into some of the
philosophy and thinking behind the design of RFC 3080. Both are short and
easy reading.
- Bruce
Re: Thoughts on an RPC protocol
Posted by Bruce Mitchener <br...@gmail.com>.
Hi Bo,
Thanks for your feedback!
On Thu, Apr 8, 2010 at 3:49 PM, Bo Shi <bs...@gmail.com> wrote:
> Hi Bruce,
>
> Would this RPC protocol take the role of the transport in the Avro
> specification or would it replace the protocol? If the handshake
> occurs on channel 0 while the request/response payloads are
> transferred on a different channel, this would not meet the existing
> wire protocol as described in the current 1.3.2 spec right?
>
This would be a transport.
I was thinking they would happen on 0 with 0 being a control channel, but
I'm not married to that. Offhand, I don't see why that would violate
anything in the specification though?
A couple other questions inline:
>
> On Thu, Apr 8, 2010 at 11:54 AM, Bruce Mitchener
> <br...@gmail.com> wrote:
> > While I recommend actually reading RFC 3080 (it is an easy read), this
> > summary may help...
> >
> > Framing: Length prefixed data, nothing unusual.
> > Encoding: Messages are effectively this:
> >
> > enum message_type {
> > message, // a request
> > reply, // when there's only a single reply
> > answer, // when there are multiple replies, send
> multiple
> > answers and then a null.
> > null, // terminate a chain of replies
> > error, // oops, there was an error
> > }
> >
> > struct message {
> > enum message_type message_type;
> > int channel;
> > int message_id;
> > bool more; // Is this message complete, or is more data
> coming?
> > for streaming
> > int sequence_number; // see RFC 3080
> > optional int answer_number; // Used for answers
> > bytes payload; // The actual RPC command, still serialized here
> > }
> >
> > When a connection is opened, there's initially one channel, channel 0.
> That
> > channel is used for commands controlling the connection state, like
> opening
> > and closing channels. We should also perform Avro RPC handshakes over
> > channel 0.
>
> Is channel 0 used exclusively as a control channel or would requests
> be allowed on this channel? Any idea on what the control messages
> would look like?
Channel 0 would be a control channel in my original proposal. I can see
arguments for allowing sending other requests on it from a simplicity point
of view. Thoughts?
Control messages would look like an Avro RPC call. They would be things
like:
OpenChannel returning the opened channel (or you could pass it the channel
number?)
CloseChannel gets passed the channel number to close.
> Channels allow for concurrency. You can send requests/messages down
> > multiple channels and process them independently. Messages on a single
> > channel need to be processed in order though. This allows for both
> > guaranteed order of execution (within a single channel) and greater
> > concurrency (multiple channels).
> >
> > Streaming happens in 2 ways.
>
> For streaming transfers, thoughts on optional compression codec
> attachment to streaming channels? It may be useful for IO-bound
> applications, but if you're transferring files like avro object
> container files that are already compressed - you'd need some extra
> coordination (but maybe that's outside the problem domain).
Compression would probably be something useful to support, agreed. That
could happen in various ways:
- Per channel (all data on the channel is compressed)
- Per request via a header of some sort
- Per connection (all data on all channels)
I suspect that I prefer per-channel from a simplicity point of view, but I'd
like to hear what people think.
- Bruce
Re: Thoughts on an RPC protocol
Posted by Bo Shi <bs...@gmail.com>.
Hi Bruce,
Would this RPC protocol take the role of the transport in the Avro
specification or would it replace the protocol? If the handshake
occurs on channel 0 while the request/response payloads are
transferred on a different channel, this would not meet the existing
wire protocol as described in the current 1.3.2 spec right?
A couple other questions inline:
On Thu, Apr 8, 2010 at 11:54 AM, Bruce Mitchener
<br...@gmail.com> wrote:
> While I recommend actually reading RFC 3080 (it is an easy read), this
> summary may help...
>
> Framing: Length prefixed data, nothing unusual.
> Encoding: Messages are effectively this:
>
> enum message_type {
> message, // a request
> reply, // when there's only a single reply
> answer, // when there are multiple replies, send multiple
> answers and then a null.
> null, // terminate a chain of replies
> error, // oops, there was an error
> }
>
> struct message {
> enum message_type message_type;
> int channel;
> int message_id;
> bool more; // Is this message complete, or is more data coming?
> for streaming
> int sequence_number; // see RFC 3080
> optional int answer_number; // Used for answers
> bytes payload; // The actual RPC command, still serialized here
> }
>
> When a connection is opened, there's initially one channel, channel 0. That
> channel is used for commands controlling the connection state, like opening
> and closing channels. We should also perform Avro RPC handshakes over
> channel 0.
Is channel 0 used exclusively as a control channel or would requests
be allowed on this channel? Any idea on what the control messages
would look like?
>
> Channels allow for concurrency. You can send requests/messages down
> multiple channels and process them independently. Messages on a single
> channel need to be processed in order though. This allows for both
> guaranteed order of execution (within a single channel) and greater
> concurrency (multiple channels).
>
> Streaming happens in 2 ways.
For streaming transfers, thoughts on optional compression codec
attachment to streaming channels? It may be useful for IO-bound
applications, but if you're transferring files like avro object
container files that are already compressed - you'd need some extra
coordination (but maybe that's outside the problem domain).
>
> The first way is to flip the more flag on a message. This means that the
> data has been broken up over multiple messages and you need to receive the
> whole thing before processing it.
>
> The second is to have multiple answers (followed by a null frame) to a
> single request message. This allows you to process the data in a streaming
> fashion. The only thing that this doesn't allow is to process the data
> being sent in a streaming fashion, but you could look at doing that by
> sending multiple request messages instead.
>
> Security and privacy can be handled by SASL.
>
> The RFC defines a number of ways in which you can detect buggy
> implementations of the protocol or invalid data being sent (framing /
> encoding violations).
>
> This should be pretty straight forward to implement, and as such (and since
> I need such a thing in the immediate future), I've already begun an
> implementation in C.
>
> - Bruce
>
> On Wed, Apr 7, 2010 at 4:13 PM, Bruce Mitchener
> <br...@gmail.com>wrote:
>
>> I'm assuming that the goals of an optimized transport for Avro RPC are
>> something like the following:
>>
>> * Framing should be efficient, easy to implement.
>> * Streaming of large values, both as part of a request and as a response
>> is very important.
>> * Being able to have multiple concurrent requests in flight, while also
>> being able to have ordering guarantees where desired is necessary.
>> * It should be easy to implement this in Java, C, Python, Ruby, etc.
>> * Security is or will be important. This security can include
>> authorization as well as privacy concerns.
>>
>> I'd like to see something based largely upon RFC 3080, with some
>> simplifications and extensions:
>>
>> http://www.faqs.org/rfcs/rfc3080.html
>>
>> What does this get us?
>>
>> * This system has mechanisms in place for streaming both a single large
>> message and breaking a single reply up into multiple answers, allowing for
>> pretty flexible streaming. (You can even mix these by having an answer that
>> gets chunked itself.)
>> * Concurrency is achieved by having multiple channels. Each channel
>> executes messages in order, so you have a good mechanism for sending
>> multiple things at once as well as maintaining ordering guarantees as
>> necessary.
>> * Reporting errors is very clear as it is a separate response type.
>> * It has already been specified pretty clearly and we'd just be evolving
>> that to something that more closely matches our needs.
>> * It specifies sufficient data that you could implement this over
>> transports other than TCP, such as UDP.
>>
>> Changes, rough list:
>>
>> * Use Avro-encoding for most things, so the encoding of a message would
>> become an Avro struct.
>> * Lose profiles in the sense that they're used in that specification since
>> we're just exchanging Avro RPCs.
>> * Do length prefixing rather than in the header, so that it is very
>> amenable to binary I/O at high volumes.
>> * No XML stuff, just existing things like the Avro handshake, wrapped up
>> in messages.
>> * For now, don't worry about things like flow control as expressed in RFC
>> 3081, mapping of 3080 to TCP.
>> * Think about adding something for true one-way messages, but an empty
>> reply frame is probably sufficient, since that still allows reporting errors
>> if needed (or desired).
>> * May well need some extensions for a more flexible security model.
>> * Use Avro RPC stuff to encode the channel management commands on channel
>> 0 rather than XML.
>>
>> RFC 3117 (http://www.faqs.org/rfcs/rfc3117.html) goes into some of the
>> philosophy and thinking behind the design of RFC 3080. Both are short and
>> easy reading.
>>
>> - Bruce
>>
>>
>
Re: Thoughts on an RPC protocol
Posted by Bruce Mitchener <br...@gmail.com>.
While I recommend actually reading RFC 3080 (it is an easy read), this
summary may help...
Framing: Length prefixed data, nothing unusual.
Encoding: Messages are effectively this:
enum message_type {
message, // a request
reply, // when there's only a single reply
answer, // when there are multiple replies, send multiple
answers and then a null.
null, // terminate a chain of replies
error, // oops, there was an error
}
struct message {
enum message_type message_type;
int channel;
int message_id;
bool more; // Is this message complete, or is more data coming?
for streaming
int sequence_number; // see RFC 3080
optional int answer_number; // Used for answers
bytes payload; // The actual RPC command, still serialized here
}
When a connection is opened, there's initially one channel, channel 0. That
channel is used for commands controlling the connection state, like opening
and closing channels. We should also perform Avro RPC handshakes over
channel 0.
Channels allow for concurrency. You can send requests/messages down
multiple channels and process them independently. Messages on a single
channel need to be processed in order though. This allows for both
guaranteed order of execution (within a single channel) and greater
concurrency (multiple channels).
Streaming happens in 2 ways.
The first way is to flip the more flag on a message. This means that the
data has been broken up over multiple messages and you need to receive the
whole thing before processing it.
The second is to have multiple answers (followed by a null frame) to a
single request message. This allows you to process the data in a streaming
fashion. The only thing that this doesn't allow is to process the data
being sent in a streaming fashion, but you could look at doing that by
sending multiple request messages instead.
Security and privacy can be handled by SASL.
The RFC defines a number of ways in which you can detect buggy
implementations of the protocol or invalid data being sent (framing /
encoding violations).
This should be pretty straight forward to implement, and as such (and since
I need such a thing in the immediate future), I've already begun an
implementation in C.
- Bruce
On Wed, Apr 7, 2010 at 4:13 PM, Bruce Mitchener
<br...@gmail.com>wrote:
> I'm assuming that the goals of an optimized transport for Avro RPC are
> something like the following:
>
> * Framing should be efficient, easy to implement.
> * Streaming of large values, both as part of a request and as a response
> is very important.
> * Being able to have multiple concurrent requests in flight, while also
> being able to have ordering guarantees where desired is necessary.
> * It should be easy to implement this in Java, C, Python, Ruby, etc.
> * Security is or will be important. This security can include
> authorization as well as privacy concerns.
>
> I'd like to see something based largely upon RFC 3080, with some
> simplifications and extensions:
>
> http://www.faqs.org/rfcs/rfc3080.html
>
> What does this get us?
>
> * This system has mechanisms in place for streaming both a single large
> message and breaking a single reply up into multiple answers, allowing for
> pretty flexible streaming. (You can even mix these by having an answer that
> gets chunked itself.)
> * Concurrency is achieved by having multiple channels. Each channel
> executes messages in order, so you have a good mechanism for sending
> multiple things at once as well as maintaining ordering guarantees as
> necessary.
> * Reporting errors is very clear as it is a separate response type.
> * It has already been specified pretty clearly and we'd just be evolving
> that to something that more closely matches our needs.
> * It specifies sufficient data that you could implement this over
> transports other than TCP, such as UDP.
>
> Changes, rough list:
>
> * Use Avro-encoding for most things, so the encoding of a message would
> become an Avro struct.
> * Lose profiles in the sense that they're used in that specification since
> we're just exchanging Avro RPCs.
> * Do length prefixing rather than in the header, so that it is very
> amenable to binary I/O at high volumes.
> * No XML stuff, just existing things like the Avro handshake, wrapped up
> in messages.
> * For now, don't worry about things like flow control as expressed in RFC
> 3081, mapping of 3080 to TCP.
> * Think about adding something for true one-way messages, but an empty
> reply frame is probably sufficient, since that still allows reporting errors
> if needed (or desired).
> * May well need some extensions for a more flexible security model.
> * Use Avro RPC stuff to encode the channel management commands on channel
> 0 rather than XML.
>
> RFC 3117 (http://www.faqs.org/rfcs/rfc3117.html) goes into some of the
> philosophy and thinking behind the design of RFC 3080. Both are short and
> easy reading.
>
> - Bruce
>
>
Re: Thoughts on an RPC protocol
Posted by Jeff Hodges <jh...@twitter.com>.
I may have misunderstood the direction this thread had taken. I'm
still going through the spec.
--
Jeff
On Sat, Apr 10, 2010 at 10:59 AM, Bruce Mitchener
<br...@gmail.com> wrote:
> What specific changes would you propose making to my proposal?
>
> - Bruce
>
> On Sat, Apr 10, 2010 at 11:57 AM, Jeff Hodges <jh...@twitter.com> wrote:
>
>> Sorry for the spam. Python, java and apache httpd implementations
>> listed at the project page: http://www.chromium.org/spdy
>>
>> On Sat, Apr 10, 2010 at 10:53 AM, Jeff Hodges <jh...@twitter.com> wrote:
>> > Oh, and it's been partially implemented in Chromium, so there's a
>> > quasi-reference implementation.
>> > --
>> > Jeff
>> >
>> > On Sat, Apr 10, 2010 at 10:48 AM, Jeff Hodges <jh...@twitter.com>
>> wrote:
>> >> To throw another set of ideas into the hat, SPDY[1][2] would be good
>> >> to learn from. SPDY takes the basics of HTTP and makes them fast.
>> >> Benefits we would enjoy include:
>> >>
>> >> * Multiplexed streams
>> >> * Request prioritization
>> >> * HTTP header compression
>> >> * Server push
>> >>
>> >> Currently in draft form.
>> >>
>> >> [1] http://dev.chromium.org/spdy/spdy-whitepaper
>> >> [2] http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft2
>> >> --
>> >> Jeff
>> >> On Fri, Apr 9, 2010 at 2:29 PM, Doug Cutting <cu...@apache.org>
>> wrote:
>> >>> Scott Carey wrote:
>> >>>>
>> >>>> I also have not wrapped my head around routing/proxy use cases. From
>> >>>> a somewhat ignorant perspective on them -- I'd rather have a solid
>> >>>> point-to-point protocol that just works, is simple, and can meet the
>> >>>> vast majority of use cases with high performance than one that
>> >>>> happens to be capable of sophisticated routing but has a lot of other
>> >>>> limitations or is a lot harder to implement and debug.
>> >>>
>> >>> FWIW, they're theoretical at this point. I was only stating that
>> prefixing
>> >>> every request and response with handshakes makes stuff like proxies
>> trivial,
>> >>> since the protocol becomes stateless. Once we start having sessions
>> things
>> >>> get trickier. For example, many HTTP client libraries cache
>> connections,
>> >>> so, if you're building on top of one of those, it's hard to know when a
>> new
>> >>> connection is opened.
>> >>>
>> >>> One approach is to declare that the current framing and handshake rules
>> only
>> >>> apply to HTTP, currently our only standard transport. Then we can
>> define a
>> >>> new transport that's point-to-point, stateful, etc. which may handle
>> framing
>> >>> and handshakes differently. Thus we can retain back-compatibility.
>> Make
>> >>> sense?
>> >>>
>> >>> Doug
>> >>>
>> >>
>> >
>>
>
Re: Thoughts on an RPC protocol
Posted by Bruce Mitchener <br...@gmail.com>.
What specific changes would you propose making to my proposal?
- Bruce
On Sat, Apr 10, 2010 at 11:57 AM, Jeff Hodges <jh...@twitter.com> wrote:
> Sorry for the spam. Python, java and apache httpd implementations
> listed at the project page: http://www.chromium.org/spdy
>
> On Sat, Apr 10, 2010 at 10:53 AM, Jeff Hodges <jh...@twitter.com> wrote:
> > Oh, and it's been partially implemented in Chromium, so there's a
> > quasi-reference implementation.
> > --
> > Jeff
> >
> > On Sat, Apr 10, 2010 at 10:48 AM, Jeff Hodges <jh...@twitter.com>
> wrote:
> >> To throw another set of ideas into the hat, SPDY[1][2] would be good
> >> to learn from. SPDY takes the basics of HTTP and makes them fast.
> >> Benefits we would enjoy include:
> >>
> >> * Multiplexed streams
> >> * Request prioritization
> >> * HTTP header compression
> >> * Server push
> >>
> >> Currently in draft form.
> >>
> >> [1] http://dev.chromium.org/spdy/spdy-whitepaper
> >> [2] http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft2
> >> --
> >> Jeff
> >> On Fri, Apr 9, 2010 at 2:29 PM, Doug Cutting <cu...@apache.org>
> wrote:
> >>> Scott Carey wrote:
> >>>>
> >>>> I also have not wrapped my head around routing/proxy use cases. From
> >>>> a somewhat ignorant perspective on them -- I'd rather have a solid
> >>>> point-to-point protocol that just works, is simple, and can meet the
> >>>> vast majority of use cases with high performance than one that
> >>>> happens to be capable of sophisticated routing but has a lot of other
> >>>> limitations or is a lot harder to implement and debug.
> >>>
> >>> FWIW, they're theoretical at this point. I was only stating that
> prefixing
> >>> every request and response with handshakes makes stuff like proxies
> trivial,
> >>> since the protocol becomes stateless. Once we start having sessions
> things
> >>> get trickier. For example, many HTTP client libraries cache
> connections,
> >>> so, if you're building on top of one of those, it's hard to know when a
> new
> >>> connection is opened.
> >>>
> >>> One approach is to declare that the current framing and handshake rules
> only
> >>> apply to HTTP, currently our only standard transport. Then we can
> define a
> >>> new transport that's point-to-point, stateful, etc. which may handle
> framing
> >>> and handshakes differently. Thus we can retain back-compatibility.
> Make
> >>> sense?
> >>>
> >>> Doug
> >>>
> >>
> >
>
Re: Thoughts on an RPC protocol
Posted by Jeff Hodges <jh...@twitter.com>.
Sorry for the spam. Python, java and apache httpd implementations
listed at the project page: http://www.chromium.org/spdy
On Sat, Apr 10, 2010 at 10:53 AM, Jeff Hodges <jh...@twitter.com> wrote:
> Oh, and it's been partially implemented in Chromium, so there's a
> quasi-reference implementation.
> --
> Jeff
>
> On Sat, Apr 10, 2010 at 10:48 AM, Jeff Hodges <jh...@twitter.com> wrote:
>> To throw another set of ideas into the hat, SPDY[1][2] would be good
>> to learn from. SPDY takes the basics of HTTP and makes them fast.
>> Benefits we would enjoy include:
>>
>> * Multiplexed streams
>> * Request prioritization
>> * HTTP header compression
>> * Server push
>>
>> Currently in draft form.
>>
>> [1] http://dev.chromium.org/spdy/spdy-whitepaper
>> [2] http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft2
>> --
>> Jeff
>> On Fri, Apr 9, 2010 at 2:29 PM, Doug Cutting <cu...@apache.org> wrote:
>>> Scott Carey wrote:
>>>>
>>>> I also have not wrapped my head around routing/proxy use cases. From
>>>> a somewhat ignorant perspective on them -- I'd rather have a solid
>>>> point-to-point protocol that just works, is simple, and can meet the
>>>> vast majority of use cases with high performance than one that
>>>> happens to be capable of sophisticated routing but has a lot of other
>>>> limitations or is a lot harder to implement and debug.
>>>
>>> FWIW, they're theoretical at this point. I was only stating that prefixing
>>> every request and response with handshakes makes stuff like proxies trivial,
>>> since the protocol becomes stateless. Once we start having sessions things
>>> get trickier. For example, many HTTP client libraries cache connections,
>>> so, if you're building on top of one of those, it's hard to know when a new
>>> connection is opened.
>>>
>>> One approach is to declare that the current framing and handshake rules only
>>> apply to HTTP, currently our only standard transport. Then we can define a
>>> new transport that's point-to-point, stateful, etc. which may handle framing
>>> and handshakes differently. Thus we can retain back-compatibility. Make
>>> sense?
>>>
>>> Doug
>>>
>>
>
Re: Thoughts on an RPC protocol
Posted by Jeff Hodges <jh...@twitter.com>.
Oh, and it's been partially implemented in Chromium, so there's a
quasi-reference implementation.
--
Jeff
On Sat, Apr 10, 2010 at 10:48 AM, Jeff Hodges <jh...@twitter.com> wrote:
> To throw another set of ideas into the hat, SPDY[1][2] would be good
> to learn from. SPDY takes the basics of HTTP and makes them fast.
> Benefits we would enjoy include:
>
> * Multiplexed streams
> * Request prioritization
> * HTTP header compression
> * Server push
>
> Currently in draft form.
>
> [1] http://dev.chromium.org/spdy/spdy-whitepaper
> [2] http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft2
> --
> Jeff
> On Fri, Apr 9, 2010 at 2:29 PM, Doug Cutting <cu...@apache.org> wrote:
>> Scott Carey wrote:
>>>
>>> I also have not wrapped my head around routing/proxy use cases. From
>>> a somewhat ignorant perspective on them -- I'd rather have a solid
>>> point-to-point protocol that just works, is simple, and can meet the
>>> vast majority of use cases with high performance than one that
>>> happens to be capable of sophisticated routing but has a lot of other
>>> limitations or is a lot harder to implement and debug.
>>
>> FWIW, they're theoretical at this point. I was only stating that prefixing
>> every request and response with handshakes makes stuff like proxies trivial,
>> since the protocol becomes stateless. Once we start having sessions things
>> get trickier. For example, many HTTP client libraries cache connections,
>> so, if you're building on top of one of those, it's hard to know when a new
>> connection is opened.
>>
>> One approach is to declare that the current framing and handshake rules only
>> apply to HTTP, currently our only standard transport. Then we can define a
>> new transport that's point-to-point, stateful, etc. which may handle framing
>> and handshakes differently. Thus we can retain back-compatibility. Make
>> sense?
>>
>> Doug
>>
>
Re: Thoughts on an RPC protocol
Posted by Jeff Hodges <jh...@twitter.com>.
Er, note that I'm generally lightly positive on using it as a
influence on the protocol, even if portions of the philosophy I find
less than ideal. I mentioned the lack of community because a lack of
active means whatever obstacles occur due to its fundamental nature
aren't well-known and well-defined. That's reasonable, if not ideal,
as we are already talking about building our own RPC!
--
Jeff
On Mon, Apr 12, 2010 at 2:48 PM, Jeff Hodges <jh...@twitter.com> wrote:
> I hadn't thought of adding SASL to SPDY. I haven't dived into the
> community, yet, to see if they've discussed it.
>
> The philosophy behind BEEP seems pretty good, even if the crazy XML
> style of the actual spec is not a good match (as put in an earlier
> email). The lack of community is worrisome, of course, but having it
> as an influence in our own design is probably worthwhile.
>
> I do worry about the desire not to "name" things. URIs seem so
> obviously a good in a system that they should be consistent for any
> implementation of the protocol and instance of its use. However, I
> recognize how much of a bite that is to chew if we don't take a
> substantial portion of a spec into our own that already has that
> defined. I'm not trying to push a REST dogma. Building a spec without
> RESTful design is fine by me, as long as we recognize the tradeoffs.
> --
> Jeff
>
> On Mon, Apr 12, 2010 at 1:46 PM, Doug Cutting <cu...@apache.org> wrote:
>> Jeff Hodges wrote:
>>>
>>> To throw another set of ideas into the hat, SPDY[1][2] would be good
>>> to learn from. SPDY takes the basics of HTTP and makes them fast.
>>> Benefits we would enjoy include:
>>>
>>> * Multiplexed streams
>>> * Request prioritization
>>> * HTTP header compression
>>> * Server push
>>>
>>> Currently in draft form.
>>>
>>> [1] http://dev.chromium.org/spdy/spdy-whitepaper
>>> [2] http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft2
>>
>> I like that SPDY is more actively developed than BEEP. It would be nice not
>> to have to re-implement clients and servers from scratch, and to perhaps
>> even use a pre-existing specfication.
>>
>> SPDY does fix one of the primary restrictions of HTTP in that it permits
>> request multiplexing: responses need not arrive in order.
>>
>> However other concerns folks have had about HTTP are that:
>> a. text headers are big and slow to process
>> b. SSL/TLS is heavyweight and inflexible for authentication
>>
>> SPDY addresses the size of headers by compressing them, but that may hinder
>> the speed of header processing.
>>
>> SPDY uses SSL/TLS, so would have the same issues there. Perhaps they could
>> be convinced to adopt SASL instead of TLS?
>>
>> Doug
>>
>
Re: Thoughts on an RPC protocol
Posted by Jeff Hodges <jh...@twitter.com>.
I hadn't thought of adding SASL to SPDY. I haven't dived into the
community, yet, to see if they've discussed it.
The philosophy behind BEEP seems pretty good, even if the crazy XML
style of the actual spec is not a good match (as put in an earlier
email). The lack of community is worrisome, of course, but having it
as an influence in our own design is probably worthwhile.
I do worry about the desire not to "name" things. URIs seem so
obviously a good in a system that they should be consistent for any
implementation of the protocol and instance of its use. However, I
recognize how much of a bite that is to chew if we don't take a
substantial portion of a spec into our own that already has that
defined. I'm not trying to push a REST dogma. Building a spec without
RESTful design is fine by me, as long as we recognize the tradeoffs.
--
Jeff
On Mon, Apr 12, 2010 at 1:46 PM, Doug Cutting <cu...@apache.org> wrote:
> Jeff Hodges wrote:
>>
>> To throw another set of ideas into the hat, SPDY[1][2] would be good
>> to learn from. SPDY takes the basics of HTTP and makes them fast.
>> Benefits we would enjoy include:
>>
>> * Multiplexed streams
>> * Request prioritization
>> * HTTP header compression
>> * Server push
>>
>> Currently in draft form.
>>
>> [1] http://dev.chromium.org/spdy/spdy-whitepaper
>> [2] http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft2
>
> I like that SPDY is more actively developed than BEEP. It would be nice not
> to have to re-implement clients and servers from scratch, and to perhaps
> even use a pre-existing specfication.
>
> SPDY does fix one of the primary restrictions of HTTP in that it permits
> request multiplexing: responses need not arrive in order.
>
> However other concerns folks have had about HTTP are that:
> a. text headers are big and slow to process
> b. SSL/TLS is heavyweight and inflexible for authentication
>
> SPDY addresses the size of headers by compressing them, but that may hinder
> the speed of header processing.
>
> SPDY uses SSL/TLS, so would have the same issues there. Perhaps they could
> be convinced to adopt SASL instead of TLS?
>
> Doug
>
Re: Thoughts on an RPC protocol
Posted by Doug Cutting <cu...@apache.org>.
Jeff Hodges wrote:
> To throw another set of ideas into the hat, SPDY[1][2] would be good
> to learn from. SPDY takes the basics of HTTP and makes them fast.
> Benefits we would enjoy include:
>
> * Multiplexed streams
> * Request prioritization
> * HTTP header compression
> * Server push
>
> Currently in draft form.
>
> [1] http://dev.chromium.org/spdy/spdy-whitepaper
> [2] http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft2
I like that SPDY is more actively developed than BEEP. It would be nice
not to have to re-implement clients and servers from scratch, and to
perhaps even use a pre-existing specfication.
SPDY does fix one of the primary restrictions of HTTP in that it permits
request multiplexing: responses need not arrive in order.
However other concerns folks have had about HTTP are that:
a. text headers are big and slow to process
b. SSL/TLS is heavyweight and inflexible for authentication
SPDY addresses the size of headers by compressing them, but that may
hinder the speed of header processing.
SPDY uses SSL/TLS, so would have the same issues there. Perhaps they
could be convinced to adopt SASL instead of TLS?
Doug
Re: Thoughts on an RPC protocol
Posted by Jeff Hodges <jh...@twitter.com>.
To throw another set of ideas into the hat, SPDY[1][2] would be good
to learn from. SPDY takes the basics of HTTP and makes them fast.
Benefits we would enjoy include:
* Multiplexed streams
* Request prioritization
* HTTP header compression
* Server push
Currently in draft form.
[1] http://dev.chromium.org/spdy/spdy-whitepaper
[2] http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft2
--
Jeff
On Fri, Apr 9, 2010 at 2:29 PM, Doug Cutting <cu...@apache.org> wrote:
> Scott Carey wrote:
>>
>> I also have not wrapped my head around routing/proxy use cases. From
>> a somewhat ignorant perspective on them -- I'd rather have a solid
>> point-to-point protocol that just works, is simple, and can meet the
>> vast majority of use cases with high performance than one that
>> happens to be capable of sophisticated routing but has a lot of other
>> limitations or is a lot harder to implement and debug.
>
> FWIW, they're theoretical at this point. I was only stating that prefixing
> every request and response with handshakes makes stuff like proxies trivial,
> since the protocol becomes stateless. Once we start having sessions things
> get trickier. For example, many HTTP client libraries cache connections,
> so, if you're building on top of one of those, it's hard to know when a new
> connection is opened.
>
> One approach is to declare that the current framing and handshake rules only
> apply to HTTP, currently our only standard transport. Then we can define a
> new transport that's point-to-point, stateful, etc. which may handle framing
> and handshakes differently. Thus we can retain back-compatibility. Make
> sense?
>
> Doug
>
Re: Thoughts on an RPC protocol
Posted by Doug Cutting <cu...@apache.org>.
Scott Carey wrote:
> I also have not wrapped my head around routing/proxy use cases. From
> a somewhat ignorant perspective on them -- I'd rather have a solid
> point-to-point protocol that just works, is simple, and can meet the
> vast majority of use cases with high performance than one that
> happens to be capable of sophisticated routing but has a lot of other
> limitations or is a lot harder to implement and debug.
FWIW, they're theoretical at this point. I was only stating that
prefixing every request and response with handshakes makes stuff like
proxies trivial, since the protocol becomes stateless. Once we start
having sessions things get trickier. For example, many HTTP client
libraries cache connections, so, if you're building on top of one of
those, it's hard to know when a new connection is opened.
One approach is to declare that the current framing and handshake rules
only apply to HTTP, currently our only standard transport. Then we can
define a new transport that's point-to-point, stateful, etc. which may
handle framing and handshakes differently. Thus we can retain
back-compatibility. Make sense?
Doug
Re: Thoughts on an RPC protocol
Posted by Scott Carey <sc...@richrelevance.com>.
On Apr 9, 2010, at 11:56 AM, Bo Shi wrote:
> On Fri, Apr 9, 2010 at 2:35 AM, Bruce Mitchener
> <br...@gmail.com> wrote:
>> Doug,
>>
>> I'm happy to hear that you like this approach!
>>
>> Allocation of channels seems to be something specific to an application. In
>> my app, I'd have a channel for the streaming data that is constantly
>> arriving and a channel for making requests on and getting back answers
>> immediately. Others could have a channel per object or whatever.
>
> One ubiquitous protocol that shares many of the same requirements and
> properties (in particular multiplexed channels over a transport) is
> SSH. Their channel mechanism may provide additional inspiration:
> [http://tools.ietf.org/html/rfc4254#section-5]. One interesting bit
> is that SSH doesn't have an explicit channel for control commands,
> instead they create additional message types for control messages that
> aren't associated with any channel. It's only a minor distinction
> though.
>
One flaw in SSH is that the bandwidth over a WAN is often pathetic.
Because it has multiple channels, it implements its own flow control and receive windows. The effective bandwidth is the window size divided by the RTT. Many implementations have a hard-coded 64KB receive buffer -- over a connection with a 30ms RTT this is peak data throughput of 2MB/sec. The latest implementation versions of SSH patch the issue by having an automatically growing buffer, but the in memory buffer size required for high throughput transfer over higher latency links is large.
http://www.docstoc.com/docs/18191581/High-Performance-Networking-with-the-SSH-Protocol/
http://www.psc.edu/networking/projects/tcptune/
http://www.psc.edu/networking/projects/hpn-ssh/
This arises out of flow control to make sure that channels do not interfere with each other too much.
HTTP does not have this problem -- it just uses TCP's flow control and its only multi-channel-like feature -- http pipelining, doesn't bother with flow control.
Whatever Avro does for a custom socket based transport should avoid this. This sort of use case will likely be common (distributed copy of HDFS across a WAN for example).
>
> If, as you suggest above, we enforce a 1-1 mapping of channel and avro
> protocol, wouldn't that eliminate the need for a handshake on
> subsequent requests on the same channel? The handshake process could
> be part of the open-channel negotiation. I'm still wrapping my head
> around the routing use-case; not sure if this meets the requirements
> there though.
I also have not wrapped my head around routing/proxy use cases. From a somewhat ignorant perspective on them -- I'd rather have a solid point-to-point protocol that just works, is simple, and can meet the vast majority of use cases with high performance than one that happens to be capable of sophisticated routing but has a lot of other limitations or is a lot harder to implement and debug.
>>>
>>
Re: Thoughts on an RPC protocol
Posted by Bo Shi <bs...@gmail.com>.
On Fri, Apr 9, 2010 at 2:35 AM, Bruce Mitchener
<br...@gmail.com> wrote:
> Doug,
>
> I'm happy to hear that you like this approach!
>
> Allocation of channels seems to be something specific to an application. In
> my app, I'd have a channel for the streaming data that is constantly
> arriving and a channel for making requests on and getting back answers
> immediately. Others could have a channel per object or whatever.
One ubiquitous protocol that shares many of the same requirements and
properties (in particular multiplexed channels over a transport) is
SSH. Their channel mechanism may provide additional inspiration:
[http://tools.ietf.org/html/rfc4254#section-5]. One interesting bit
is that SSH doesn't have an explicit channel for control commands,
instead they create additional message types for control messages that
aren't associated with any channel. It's only a minor distinction
though.
>
> Are your proxy servers custom software or are they just passing traffic
> along directly? If they're Avro-aware, then they can manage the handshaking
> process when routing to a new peer. Is this something that is actively
> happening today or just something that is possible?
>
> I definitely agree about not wanting a handshake per request. For my
> application that would add a lot of overhead in terms of the data
> transmitted. (I'm sending a lot of small requests, hopefully many thousands
> per second...) I would be much much happier being able to have a handshake
> per connection (or per channel open).
>
If, as you suggest above, we enforce a 1-1 mapping of channel and avro
protocol, wouldn't that eliminate the need for a handshake on
subsequent requests on the same channel? The handshake process could
be part of the open-channel negotiation. I'm still wrapping my head
around the routing use-case; not sure if this meets the requirements
there though.
> - Bruce
>
> On Thu, Apr 8, 2010 at 4:43 PM, Doug Cutting <cu...@apache.org> wrote:
>
>> Bruce,
>>
>> Overall this looks like a good approach to me.
>>
>> How do you anticipate allocating channels? I'm guessing this would be one
>> per client object, that a pool of open connections to servers would be
>> maintained, and creating a new client object would allocate a new channel.
>>
>> Currently we perform a handshake per request. This is fairly cheap and
>> permits things like routing through proxy servers. Different requests over
>> the same connection can talk to different backend servers running different
>> versions of the protocol. Also consider the case where, between calls on an
>> object, the connection times out, and a new session is established and a new
>> handshake must take place.
>>
>> That said, having a session where the handshake can be assumed vastly
>> simplifies one-way messages. Without a response or error on which to prefix
>> a handshake response, a one-way client has no means to know that the server
>> was able to even parse its request. Yet we'd still like a handshake for
>> one-way messages, so that clients and servers need not be versioned in
>> lockstep. So the handshake-per-request model doesn't serve one-way messages
>> well.
>>
>> How can we address both of these needs: to permit flexible payload routing
>> and efficient one-way messaging?
>>
>> Doug
>>
>>
>> Bruce Mitchener wrote:
>>
>>> * Think about adding something for true one-way messages, but an empty
>>> reply frame is probably sufficient, since that still allows reporting
>>> errors
>>> if needed (or desired).
>>>
>>
>
Re: Thoughts on an RPC protocol
Posted by Bruce Mitchener <br...@gmail.com>.
On Fri, Apr 9, 2010 at 10:00 AM, Scott Carey <sc...@richrelevance.com>wrote:
>
> On Apr 8, 2010, at 11:35 PM, Bruce Mitchener wrote:
>
> > Doug,
> >
> > I'm happy to hear that you like this approach!
> >
> > Allocation of channels seems to be something specific to an application.
> In
> > my app, I'd have a channel for the streaming data that is constantly
> > arriving and a channel for making requests on and getting back answers
> > immediately. Others could have a channel per object or whatever.
>
> If this is all on one TCP port, then channels will interfere with one
> another somewhat -- the transport layer will see packets arrive in the order
> they were sent. If one packet in your streaming data stalls, both channels
> will stall. Depending on the application requirements, this might be fine.
> But it should be made clear that channels are not independent, they are
> just interleaved over one ordered data stream. How each implementation
> orders sending data on one end will affect order on the other side.
Agreed, that's just a fact of life with TCP. Perhaps if SCTP ever gets some
traction, then people can do a mapping for that. In the meantime, we could
look at what RFC 3081 did in the TCP mapping for RFC 3080 with respect to
flow control.
> I definitely agree about not wanting a handshake per request. For my
> > application that would add a lot of overhead in terms of the data
> > transmitted. (I'm sending a lot of small requests, hopefully many
> thousands
> > per second...) I would be much much happier being able to have a
> handshake
> > per connection (or per channel open).
> >
>
> Handshake per request will limit WAN usage. Doubling request latency isn't
> a problem for local networks with sub 0.1ms RTTs, but it is a problem with
> 25ms RTTs. Round trips aren't free on the processing or bandwidth side
> either. If there is a way to meet most goals and limit extra handshakes to
> specific cases that would be a significant performance improvement.
We agree very strongly here.
- Bruce
Re: Thoughts on an RPC protocol
Posted by Scott Carey <sc...@richrelevance.com>.
On Apr 8, 2010, at 11:35 PM, Bruce Mitchener wrote:
> Doug,
>
> I'm happy to hear that you like this approach!
>
> Allocation of channels seems to be something specific to an application. In
> my app, I'd have a channel for the streaming data that is constantly
> arriving and a channel for making requests on and getting back answers
> immediately. Others could have a channel per object or whatever.
If this is all on one TCP port, then channels will interfere with one another somewhat -- the transport layer will see packets arrive in the order they were sent. If one packet in your streaming data stalls, both channels will stall. Depending on the application requirements, this might be fine. But it should be made clear that channels are not independent, they are just interleaved over one ordered data stream. How each implementation orders sending data on one end will affect order on the other side.
>
> I definitely agree about not wanting a handshake per request. For my
> application that would add a lot of overhead in terms of the data
> transmitted. (I'm sending a lot of small requests, hopefully many thousands
> per second...) I would be much much happier being able to have a handshake
> per connection (or per channel open).
>
Handshake per request will limit WAN usage. Doubling request latency isn't a problem for local networks with sub 0.1ms RTTs, but it is a problem with 25ms RTTs. Round trips aren't free on the processing or bandwidth side either. If there is a way to meet most goals and limit extra handshakes to specific cases that would be a significant performance improvement.
> - Bruce
>
> On Thu, Apr 8, 2010 at 4:43 PM, Doug Cutting <cu...@apache.org> wrote:
>
>> Bruce,
>>
>> Overall this looks like a good approach to me.
>>
>> How do you anticipate allocating channels? I'm guessing this would be one
>> per client object, that a pool of open connections to servers would be
>> maintained, and creating a new client object would allocate a new channel.
>>
>> Currently we perform a handshake per request. This is fairly cheap and
>> permits things like routing through proxy servers. Different requests over
>> the same connection can talk to different backend servers running different
>> versions of the protocol. Also consider the case where, between calls on an
>> object, the connection times out, and a new session is established and a new
>> handshake must take place.
>>
>> That said, having a session where the handshake can be assumed vastly
>> simplifies one-way messages. Without a response or error on which to prefix
>> a handshake response, a one-way client has no means to know that the server
>> was able to even parse its request. Yet we'd still like a handshake for
>> one-way messages, so that clients and servers need not be versioned in
>> lockstep. So the handshake-per-request model doesn't serve one-way messages
>> well.
>>
>> How can we address both of these needs: to permit flexible payload routing
>> and efficient one-way messaging?
>>
>> Doug
>>
>>
>> Bruce Mitchener wrote:
>>
>>> * Think about adding something for true one-way messages, but an empty
>>> reply frame is probably sufficient, since that still allows reporting
>>> errors
>>> if needed (or desired).
>>>
>>
Re: Thoughts on an RPC protocol
Posted by Bruce Mitchener <br...@gmail.com>.
Doug,
I'm happy to hear that you like this approach!
Allocation of channels seems to be something specific to an application. In
my app, I'd have a channel for the streaming data that is constantly
arriving and a channel for making requests on and getting back answers
immediately. Others could have a channel per object or whatever.
Are your proxy servers custom software or are they just passing traffic
along directly? If they're Avro-aware, then they can manage the handshaking
process when routing to a new peer. Is this something that is actively
happening today or just something that is possible?
I definitely agree about not wanting a handshake per request. For my
application that would add a lot of overhead in terms of the data
transmitted. (I'm sending a lot of small requests, hopefully many thousands
per second...) I would be much much happier being able to have a handshake
per connection (or per channel open).
- Bruce
On Thu, Apr 8, 2010 at 4:43 PM, Doug Cutting <cu...@apache.org> wrote:
> Bruce,
>
> Overall this looks like a good approach to me.
>
> How do you anticipate allocating channels? I'm guessing this would be one
> per client object, that a pool of open connections to servers would be
> maintained, and creating a new client object would allocate a new channel.
>
> Currently we perform a handshake per request. This is fairly cheap and
> permits things like routing through proxy servers. Different requests over
> the same connection can talk to different backend servers running different
> versions of the protocol. Also consider the case where, between calls on an
> object, the connection times out, and a new session is established and a new
> handshake must take place.
>
> That said, having a session where the handshake can be assumed vastly
> simplifies one-way messages. Without a response or error on which to prefix
> a handshake response, a one-way client has no means to know that the server
> was able to even parse its request. Yet we'd still like a handshake for
> one-way messages, so that clients and servers need not be versioned in
> lockstep. So the handshake-per-request model doesn't serve one-way messages
> well.
>
> How can we address both of these needs: to permit flexible payload routing
> and efficient one-way messaging?
>
> Doug
>
>
> Bruce Mitchener wrote:
>
>> * Think about adding something for true one-way messages, but an empty
>> reply frame is probably sufficient, since that still allows reporting
>> errors
>> if needed (or desired).
>>
>
Re: Thoughts on an RPC protocol
Posted by Jeremy Custenborder <jc...@gmail.com>.
I really like the model that Voldemort uses for their protocol buffers
implementation. I ported their client to .net and it was really
simple. The framing is really simple as it prefixes an integer value
before binary data. The binary data is a request made with a protocol
buffer, and so is the response. The response is a protocol buffer
package specific to the method called. They used enums for the method
names. For streaming results it prefixes a 1 if there is another
record and either a 0 or -1 (I forget and need to look at the code.
You get the idea) if it's the end of the stream. I like the idea of
keeping things as simple as possible because it makes it easier for
additional languages to be added quickly. Personally I would prefer to
have an easy request then response model that blocks on the client
side. For simplicity a client could just block and wait for the data
to return or the client could be more advanced and use callbacks or io
completion ports. This would allow a client to use things like
connection pooling to handle concurrency of multiple requests.
As for routing I typically don't like using a proxy because that
limits you to the interface bandwidth of the proxy appliance. For
example if you use something like a netscaler as your proxy in front
of your back end servers. You will have 50 back end servers with gig
connections trying to feed a couple netscalers that can constrain the
bandwidth. Adding more bandwidth means adding more proxy boxes which
gets expensive fast especially with netscalers. Voldemort uses the
concept of node banning. If a node doesn't respond quickly enough or
has errors it will get banned for a period of time. Couple this with
something like SRV records in DNS and you can easily manipulate the
direction of your traffic without using a proxy saving you some cash.
I'm currently working on the .net port of avro and the rpc
implementation is one of my priorities. My goal is to get something to
the point where I can have a utility that connects to the rpc server,
gets protocol handshake, then builds out strongly typed code that a
developer can work against.
{
"namespace": "com.acme",
"protocol": "HelloWorld",
"doc": "Protocol Greetings",
"types": [
{"name": "Greeting", "type": "record", "fields": [
{"name": "message", "type": "string"}]},
{"name": "Curse", "type": "error", "fields": [
{"name": "message", "type": "string"}]}
],
"messages": {
"hello": {
"doc": "Say hello.",
"request": [{"name": "greeting", "type": "Greeting" }],
"response": "Greeting",
"errors": ["Curse"]
}
}
}
would generate
namespace com.acme
{
public class HelloWorld:Avro.Protocol
{
public Greeting hello(Greeting greeting)
}
}
On Thu, Apr 8, 2010 at 3:43 PM, Doug Cutting <cu...@apache.org> wrote:
> Bruce,
>
> Overall this looks like a good approach to me.
>
> How do you anticipate allocating channels? I'm guessing this would be one
> per client object, that a pool of open connections to servers would be
> maintained, and creating a new client object would allocate a new channel.
>
> Currently we perform a handshake per request. This is fairly cheap and
> permits things like routing through proxy servers. Different requests over
> the same connection can talk to different backend servers running different
> versions of the protocol. Also consider the case where, between calls on an
> object, the connection times out, and a new session is established and a new
> handshake must take place.
>
> That said, having a session where the handshake can be assumed vastly
> simplifies one-way messages. Without a response or error on which to prefix
> a handshake response, a one-way client has no means to know that the server
> was able to even parse its request. Yet we'd still like a handshake for
> one-way messages, so that clients and servers need not be versioned in
> lockstep. So the handshake-per-request model doesn't serve one-way messages
> well.
>
> How can we address both of these needs: to permit flexible payload routing
> and efficient one-way messaging?
>
> Doug
>
> Bruce Mitchener wrote:
>>
>> * Think about adding something for true one-way messages, but an empty
>> reply frame is probably sufficient, since that still allows reporting
>> errors
>> if needed (or desired).
>
Re: Thoughts on an RPC protocol
Posted by Doug Cutting <cu...@apache.org>.
Bruce,
Overall this looks like a good approach to me.
How do you anticipate allocating channels? I'm guessing this would be
one per client object, that a pool of open connections to servers would
be maintained, and creating a new client object would allocate a new
channel.
Currently we perform a handshake per request. This is fairly cheap and
permits things like routing through proxy servers. Different requests
over the same connection can talk to different backend servers running
different versions of the protocol. Also consider the case where,
between calls on an object, the connection times out, and a new session
is established and a new handshake must take place.
That said, having a session where the handshake can be assumed vastly
simplifies one-way messages. Without a response or error on which to
prefix a handshake response, a one-way client has no means to know that
the server was able to even parse its request. Yet we'd still like a
handshake for one-way messages, so that clients and servers need not be
versioned in lockstep. So the handshake-per-request model doesn't serve
one-way messages well.
How can we address both of these needs: to permit flexible payload
routing and efficient one-way messaging?
Doug
Bruce Mitchener wrote:
> * Think about adding something for true one-way messages, but an empty
> reply frame is probably sufficient, since that still allows reporting errors
> if needed (or desired).