Re: Using Avro with something like OSGi or NetBeans RCP; modular Java systems dealing with classloader separation

[Wade Chandler <hw...@yahoo.com>]

>________________________________
>From: Doug Cutting <cu...@apache.org>
>To: user@avro.apache.org
>Sent: Friday, February 18, 2011 7:18 PM
>Subject: Re: Using Avro with something like OSGi or NetBeans RCP; modular Java systems dealing with classloader separation

Doug,

It has been a while since you replied, and I am just now finally getting back to Avro for some things I'm doing. Thanks for the information by the way. The rest is inline...


>
>On 02/18/2011 01:16 PM, Wade Chandler wrote:
>> I have had different thoughts. I'm wanting to use as much of Avro which is
>> already available as possible. The top two seem to be:
>>
>> 1) Write my own server implementation which handles multiple responders.
>
>This seems like a sub-optimal approach.
>
>> 2) Write a responder implementation which takes multiple providers and builds a
>> unified protocol to be parsed.
>
>This sounds better to me.
>
>You might be able to use SpecificResponder directly if you construct a
>java.lang.reflect.Proxy that implements all of the module interfaces and
>whose InvocationHandler dispatches to the appropriate module
>implementation.  So this might look something like:
>
>public static Object createProxy(Class[] interfaces, Object[] impls);
>
>This proxy could then be passed to SpecificResponder as the
>implementation.  You'd then also need to construct a protocol that
>appends the messages of all of the modules.  Note however that with this
>approach no two modules could contain messages of the same name.
>

This definitely sounds doable on the server side. Similarly I suppose I could do something similar on the client with the requester. Something I have been thinking about though the more I look at this is that it would be extremely handy if RPC would allow multiple protocols. I know that would mean quite a few languages would have to be changed for such a thing, but seems that wouldn't be too hard since the general mechanics are already there.


I looked in Jira for such a request and didn't see one. Any such plans for 1.6? Would you guys be open to such support being added? I have been thinking about forking, just to get it worked in, and then seeing what folks thought about it and trying to get that worked back into Avro. My fork would be private until such a thing took place as I wouldn't want a competing project, but would really just want to use it for some of my own things until it was an official piece of the project.

Right now, to get something similar working without multiple protocol support and thus having different namespaces which I can use to herd messages I have a couple ideas:

1) Based on your suggestion use a proxy and merge responder interfaces together. A generic responder could then direct traffic based on the message name (as the protocol would only have a single namespace) to the correct implementation. Do something similar on the client side for the requester. This assuming there would be a modular client as well. In this situation I need to parse the protocol files of each module to merge them into a single unified protocol. The protocols all become part of the applications namespace and the schemas may still retain their original namespaces.


2) Define a general protocol contract at the application level. All responders will implement the same protocol; which for the sake of code generation could have different namespaces. Not having methods matching the correct signature will be an error, and thus I expect them all to have the same exact interface. There will be a generic request and a response which simply takes some generic object. The different modules schemas will be merged. The requests schema objects namespace will be used to direct traffic. The schemas technically become the requests and replies; data in and data out.

Both seem doable with Avro today albeit they just don't feel "right". I'm very interested in multiple protocol support if there is any chance such support being added would be accepted unless it is already on the list of things to do; in that case I would gladly help out if I can.


>> When modules are perhaps live updated in the
>> server send some message telling all clients that connections must be
>> reconnected unless the updates require client updates which can force them to
>> restart in which case they need to reconnect anyways. After this message has
>> been sent out, close all connections from the server side, rebuild the protocol
>> from available providers, restart the server. Clients will have a period of time
>> before they timeout after trying once receiving the message.
>
>Could you simply restart the server, closing all client connections when
>they complete their currently executing request, without sending any
>special message to the clients?  Then the clients would simply need to
>be written to retry requests when they get a connection closed exception.
>

Yes, I think that would work fine.


>> The snag here seems to be on the client side. It isn't straight forward exactly
>> how the client protocols which were merged on the server would react to this.
>> Perhaps it isn't a big deal if some how versions are the same and the records
>> and messages match up. Not sure exactly as I'm just beginning.
>
>The responder doesn't currently check that the client and server
>protocol names match, so this should mostly just work.  The only problem
>I see is if two protocols have a message with the same name (as
>mentioned above).  If you need to permit that, then you couldn't use the
>proxy approach above, but would need to use a responder that wraps or
>extends SpecificResponder and dispatches to the right implementation
>instance based on the protocol name, not just the message name.  The
>client's protocol name is available through a ThreadLocal as
>Responder.getRemote().
>


Seems in this situation I would have to have specific client connections versus all messages traveling over a single connection with merged protocols. i.e. Each individual client connection performs its own hand shake and thus each one is using a specific protocol on the connection. Is that correct? Well, unless I do as we talked above on the client side as well.


This too makes me feel like a better solution would be to allow multiple protocols to be used for RPC. Then a simple wrapping responder or requester could handle directing the messages to the appropriate specific instances. Seems a good addition to the API. Seems outside of the multiple protocol support, the logic already there could be used to create specialized requesters/responders which

proxy to others based off message names and their protocols; well, the C/C++ versions may need some special things to return method pointers or something, but generally I think that is fairly accurate.

Thanks again for the advice and help, and thanks to you and all other devs for Avro, I like it much better than protocol buffers,

Wade

 
==================
Wade Chandler
Software Engineer and Developer
NetBeans Dream Team Member and Contributor


http://wiki.netbeans.org/wiki/view/NetBeansDreamTeam
http://www.netbeans.org

Re: Using Avro with something like OSGi or NetBeans RCP; modular Java systems dealing with classloader separation

[Doug Cutting <cu...@apache.org>]

On 10/04/2011 02:49 PM, Wade Chandler wrote:
> I'm very interested in multiple protocol support if there is any
> chance such support being added would be accepted unless it is
> already on the list of things to do; in that case I would gladly help
> out if I can.

I don't know of anyone currently working on this.  A good implementation
would be welcome.

> Seems in this situation I would have to have specific client
> connections versus all messages traveling over a single connection
> with merged protocols. i.e. Each individual client connection
> performs its own hand shake and thus each one is using a specific
> protocol on the connection. Is that correct?

If you use HTTP then each protocol can be at a different URL.  HTTP
clients pool connections to a given server, so that different protocols
can share a single connection.  If HTTP is suitable for your RPC needs
then you might not need to implement anything new.  Other transceiver
implementations do not currently pool connections but represent a
dedicated connection ("stateful" in the terminology of Avro's
specification.)

> This too makes me feel like a better solution would be to allow
> multiple protocols to be used for RPC.

A simple way to implement this might be to make other Transceivers be
"stateless" like HttpTransceiver.  (This is easy to implement: just
override isConnected to always return false.)  Then every client request
will be prefixed by the protocol handshake, uniquely identifying the
protocol.  (The handshake is only a pair of 16-byte MD5 hash codes, so
only adds ~32 bytes to each request.)  Existing client code should not
otherwise need to be changed.  Stateless Transceivers should already be
able to be used by multiple Requestors.

The primary change would be on the server side.  Responder's constructor
would need to accept a list of multiple protocols.  Responder#respond()
could be passed the remote protocol.  SpecificResponder would accept a
list of implementations and dispatch based on the handshake.

Doug