You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@river.apache.org by Peter Firmstone <ji...@zeus.net.au> on 2013/01/05 10:03:18 UTC
Discovery v3
Gregg,
Now you've got me thinking about Discovery v3 ;)
replace hostname and port with a RFC3986 uri string.
expand the URI format to allow other sockets, for example:
* jini://hostname:port#IP - TCP/IP Socket
* jini://hostid#USB- USB Socket
* jini://hostname:port?socID=500#UDT - UDT Socket with socketID for
multiplexing 10x faster than TCP sockets and firewall friendly.
* jini://hostname:port#MEKONG - Sim?
replace SHA-1 hash functions with SHA-2 in |net.jini.discovery.ssl| format.
Cheers,
Peter.
Re: Discovery v3
Posted by Peter Firmstone <ji...@zeus.net.au>.
Thanks Dan, I'd like to get the semantics right.
Cheers,
Peter.
On 9/01/2013 7:51 AM, Dan Creswell wrote:
>> It sounds like the socket provider is a separate concen from the uri string, to enable substitution of socket providers.
>>
> Not so sure. See there's protocol and transport for the protocol. One
> needs both in order to get a connection and do stuff with it. That
> means that whilst there might be a library of socket providers, and a
> library of protocols, one can't "cross" any with any meaningfully
> because of standard conventions enforced at firewalls and so on.
>
> So I reckon we need something that can specific the whole stack,
> whether we like it or not. There will of course be "obvious" defaults
> for various situations, again defined by convention.
>
> Gonna take some thought I reckon....not something to hurry into. I'm
> gonna have to sit back and see if I can think of some prior art in
> this area to look at.
>
>> SocketAddress is an abstract class and Sockets are just endpoints for communication between two machines.
>>
>> Sockets don't need to be trusted, simplifying discovery trust issues, trust is left to the jeri endpoint used over that socket.
>>
>> Cheers,
>>
>> Peter.
>>
>>
Re: Discovery v3
Posted by Dan Creswell <da...@gmail.com>.
>
> It sounds like the socket provider is a separate concen from the uri string, to enable substitution of socket providers.
>
Not so sure. See there's protocol and transport for the protocol. One
needs both in order to get a connection and do stuff with it. That
means that whilst there might be a library of socket providers, and a
library of protocols, one can't "cross" any with any meaningfully
because of standard conventions enforced at firewalls and so on.
So I reckon we need something that can specific the whole stack,
whether we like it or not. There will of course be "obvious" defaults
for various situations, again defined by convention.
Gonna take some thought I reckon....not something to hurry into. I'm
gonna have to sit back and see if I can think of some prior art in
this area to look at.
> SocketAddress is an abstract class and Sockets are just endpoints for communication between two machines.
>
> Sockets don't need to be trusted, simplifying discovery trust issues, trust is left to the jeri endpoint used over that socket.
>
> Cheers,
>
> Peter.
>
>
Re: Discovery v3
Posted by Peter Firmstone <ji...@zeus.net.au>.
On 9/01/2013 3:48 AM, Gregg Wonderly wrote:
> I'm not convinced it's that "old". It's been around for eternity, but from a practical perspective, there's something to be said for consistency.
But "jini" is registered with IANA, it's also registered as a dns-srv.
This is from Wikipedia:
URI schemes are frequently and incorrectly referred to as
"protocols", or specifically as *URI protocols* or *URL
<http://en.wikipedia.org/wiki/URL> protocols*, since most were
originally designed to be used with a particular protocol
<http://en.wikipedia.org/wiki/Protocol_%28computing%29>, and often
have the same name. The |http| scheme, for instance, is generally
used for interacting with Web resources
<http://en.wikipedia.org/wiki/Resource_%28Web%29> using HyperText
Transfer Protocol
<http://en.wikipedia.org/wiki/HyperText_Transfer_Protocol>. Today,
URIs with that scheme are also used for other purposes, such as RDF
<http://en.wikipedia.org/wiki/Resource_Description_Framework>
resource identifiers and XML namespaces
<http://en.wikipedia.org/wiki/XML_namespaces>, that are not related
to the protocol. Furthermore, some URI schemes are not associated
with any specific protocol (e.g. "|file
<http://en.wikipedia.org/wiki/File_URI_scheme>|") and many others do
not use the name of a protocol as their prefix (e.g. "|news
<http://en.wikipedia.org/wiki/Usenet_newsgroup>|").
URI schemes should be registered with IANA
<http://en.wikipedia.org/wiki/Internet_Assigned_Numbers_Authority>,
although non-registered schemes are used in practice. RFC 4395
<http://tools.ietf.org/html/rfc4395> describes the procedures for
registering new URI schemes
The jini scheme already has existing methods for selecting different
protocols, but it's not included in the url, perhaps that's a shortcoming.
A guide for creating new uri / url sheme's:
http://www.ietf.org/rfc/rfc2718.txt
>> A size limit of 512 bytes is recommended.
> This is multicast, and I'm not worrying about multicast, it already works, what we want to do, is something completely different which has different network characteristics such as firewall traversal, interaction with new Java modularization, different forms of security etc.
Some deployments still use multicast, which depend on unicast, we need
to support that a little longer. Sounds like we might be getting our
wires crossed, I'm discussing modifying multicast packets (which must
use unicast) to include some additional information to assist unicast on
the client to determine the most suitable socket for the Endpoint to use
for unicast discovery.
You might recall Sim added a SocketFactory to LookupLocator.
LookupLocator is a value object, it didn't fit, I'd like to provide the
same functionality via alternate means.
> This is where we need to "reconsider" what is going on. Endpoints are a fine abstraction, but we want to make it possible for the protocol handler to end up with the appropriate EndPoint abstraction that supports the protocol requirements.
>
Unicast lookup discovery currently uses Endpoints. All existing
Endpoints accept a SocketFactory, however discovery doesn't use it,
instead a standard socket is used, this isn't configurable.
The Endpoint used for unicast discovery can be configured.
ConstrainableLookupLocator can ensure that the right kind of Endpoint is
used.
Are you looking for a more direct way, to use your preferred Endpoint
for unicast discovery? You want your url string to specify which
endpoint to use?
Perhaps we could do that with a query string:
jini://host:port?endpoint=JERI-SSL
jini://host:port?endpoint=JRMP
jini://host:port?endpoint=RMI-IIOP
And perhaps extending it just that little bit further:
jini://host:port?endpoint=JERI-SSL,socket=MEKONG
Cheers,
Peter.
Re: Discovery v3
Posted by Gregg Wonderly <gr...@gmail.com>.
On Jan 8, 2013, at 2:02 AM, Peter <ji...@zeus.net.au> wrote:
> That's old school mate, in uri, shemes aren't protocols, that's historical. Schemes should also be registered.
I'm not convinced it's that "old". It's been around for eternity, but from a practical perspective, there's something to be said for consistency.
>
> There's something more going on here, we need to think it over.
>
> Lets look at how the discovery protocol works now (somewhat simplified):
>
> Your client, using multicast discovery recieves a udp version 2 packet, it contains the following:
>
> Protocol version
> multicast packet type
> discovery format id
> discovery format data (bytes)
>
> The discovery format data encompasses all data from the end of the format id to the end of the packet and is encoded according to the discovery format indicated by the discovery format id. Which contains:
>
> Host
> port
> lookup service id
> member groups
> sequence number.
>
> A size limit of 512 bytes is recommended.
This is multicast, and I'm not worrying about multicast, it already works, what we want to do, is something completely different which has different network characteristics such as firewall traversal, interaction with new Java modularization, different forms of security etc.
>
> The client then performs unicast discovery using a tcp Socket, with the hostname and port provided.
That's what "Jini:" specifies, yes, but that's where we are thinking about doing something completely different than what we already have.
>
> So far we're considering replacing the host and port with a uri string.
>
> But there's a need also to allow custom Socket's as Sim demonstrated with disco v1 and LookupLocator.
Custom sockets are part of the "protocol handler". It will be pluggable at the client end, and thus creation of the socket connectivity (or whatever it is) can be configurable in that way.
> URL handlers aren't suitable, they only perform retrieval, we need a bi directional socket connection.
The existing URLStreamHandler provides the ability to "create the connection" that's all. We just need to create a similar mechanism which provides for "stream" based I/O in both directions. I think that the EndPoint mechanism is the right place to look/hookup.
> It sounds like the socket provider is a separate concen from the uri string, to enable substitution of socket providers.
Sockets should be provided in the way that the "protocol" requires. I'm not convince that these are separable bits. At the most, the protocol handler should provide an API for InputStream and OutputStream setup. Socket is a very limiting thing. NIO should also be part of what happens.
> SocketAddress is an abstract class and Sockets are just endpoints for communication between two machines.
Sockets are one type of endpoint, not the only type...
> Sockets don't need to be trusted, simplifying discovery trust issues, trust is left to the jeri endpoint used over that socket.
This is where we need to "reconsider" what is going on. Endpoints are a fine abstraction, but we want to make it possible for the protocol handler to end up with the appropriate EndPoint abstraction that supports the protocol requirements.
Gregg
>
> Cheers,
>
> Peter.
>
>
> ----- Original message -----
>> Humm, that "protocol" specifier has, in all current usages of URI/URL that I've
>> got experience with, specific transport included in the protocol. HTTP, is an
>> application protocol, which assumes a "streaming socket" connection to TCP port
>> 80. Other application protocols have similar details. HTTPS is used for ssl
>> encapsulation of the "streaming socket" connection. It imposes a different,
>> default TCP socket (443), since the actual transport negotiation, which occurs
>> up front, needs to progress differently than port 80.
>>
>> I don't think there is any practical reason why the protocol/scheme name can't
>> be used to select a driver/stream-handler like software implementation. The
>> protocol handler will have the scheme of the URL will it not?
>>
>> Gregg Wonderly
>>
>> On Jan 7, 2013, at 4:42 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
>>
>>> On 7/01/2013 11:32 AM, Gregg Wonderly wrote:
>>>> I think that using "reference" rather than a protocol string is not a good
>>>> idea, because it puts this important differentiation at the end rather than
>>>> first. Just using
>>>>
>>>> jini: for discovery V1 or V2
>>>> usb: for something over a USB interface of yet to be decided nature
>>>> udt: for UDT
>>>> mekong: for mekong
>>>>
>>>> etc., would be the right choice. For me, that would provide a better bases
>>>> for a protocol handler/driver mechanism.
>>>>
>>>> Gregg Wonderly
>>>
>>> I did consider that, but it clobbers dns, we loose the scheme information,
>>> making life harder if we wanted to do dns-srv discovery. (Not that you'll use
>>> dns-srv for usb, but you might for mekong or udt).
>>>
>>> Someone using dns srv to discover udt won't know whether it's jini or
>>> something else.
>>>
>>> You're right though, using the fragment or reference component is a hack.
>>>
>>> We could use the path segment, it has no meaning presently for discovery,
>>> jini://.
>>>
>>> "/" could potentially be used as a separator for nested Socket providers.
>>>
>>> This might allow the client and server to nest Sockets in an identical
>>> hierarchy.
>>>
>>> discovery SSL
>>> |
>>> DeflateSocket (compression)
>>> |
>>> MekongSocket
>>>
>>> Cheers,
>>>
>>> Peter.
>>>
>>
>
Re: Discovery v3
Posted by Peter <ji...@zeus.net.au>.
That's old school mate, in uri, shemes aren't protocols, that's historical. Schemes should also be registered.
There's something more going on here, we need to think it over.
Lets look at how the discovery protocol works now (somewhat simplified):
Your client, using multicast discovery recieves a udp version 2 packet, it contains the following:
Protocol version
multicast packet type
discovery format id
discovery format data (bytes)
The discovery format data encompasses all data from the end of the format id to the end of the packet and is encoded according to the discovery format indicated by the discovery format id. Which contains:
Host
port
lookup service id
member groups
sequence number.
A size limit of 512 bytes is recommended.
The client then performs unicast discovery using a tcp Socket, with the hostname and port provided.
So far we're considering replacing the host and port with a uri string.
But there's a need also to allow custom Socket's as Sim demonstrated with disco v1 and LookupLocator.
URL handlers aren't suitable, they only perform retrieval, we need a bi directional socket connection.
It sounds like the socket provider is a separate concen from the uri string, to enable substitution of socket providers.
SocketAddress is an abstract class and Sockets are just endpoints for communication between two machines.
Sockets don't need to be trusted, simplifying discovery trust issues, trust is left to the jeri endpoint used over that socket.
Cheers,
Peter.
----- Original message -----
> Humm, that "protocol" specifier has, in all current usages of URI/URL that I've
> got experience with, specific transport included in the protocol. HTTP, is an
> application protocol, which assumes a "streaming socket" connection to TCP port
> 80. Other application protocols have similar details. HTTPS is used for ssl
> encapsulation of the "streaming socket" connection. It imposes a different,
> default TCP socket (443), since the actual transport negotiation, which occurs
> up front, needs to progress differently than port 80.
>
> I don't think there is any practical reason why the protocol/scheme name can't
> be used to select a driver/stream-handler like software implementation. The
> protocol handler will have the scheme of the URL will it not?
>
> Gregg Wonderly
>
> On Jan 7, 2013, at 4:42 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
>
> > On 7/01/2013 11:32 AM, Gregg Wonderly wrote:
> > > I think that using "reference" rather than a protocol string is not a good
> > > idea, because it puts this important differentiation at the end rather than
> > > first. Just using
> > >
> > > jini: for discovery V1 or V2
> > > usb: for something over a USB interface of yet to be decided nature
> > > udt: for UDT
> > > mekong: for mekong
> > >
> > > etc., would be the right choice. For me, that would provide a better bases
> > > for a protocol handler/driver mechanism.
> > >
> > > Gregg Wonderly
> >
> > I did consider that, but it clobbers dns, we loose the scheme information,
> > making life harder if we wanted to do dns-srv discovery. (Not that you'll use
> > dns-srv for usb, but you might for mekong or udt).
> >
> > Someone using dns srv to discover udt won't know whether it's jini or
> > something else.
> >
> > You're right though, using the fragment or reference component is a hack.
> >
> > We could use the path segment, it has no meaning presently for discovery,
> > jini://.
> >
> > "/" could potentially be used as a separator for nested Socket providers.
> >
> > This might allow the client and server to nest Sockets in an identical
> > hierarchy.
> >
> > discovery SSL
> > |
> > DeflateSocket (compression)
> > |
> > MekongSocket
> >
> > Cheers,
> >
> > Peter.
> >
>
Re: Discovery v3
Posted by Gregg Wonderly <gr...@gmail.com>.
Humm, that "protocol" specifier has, in all current usages of URI/URL that I've got experience with, specific transport included in the protocol. HTTP, is an application protocol, which assumes a "streaming socket" connection to TCP port 80. Other application protocols have similar details. HTTPS is used for ssl encapsulation of the "streaming socket" connection. It imposes a different, default TCP socket (443), since the actual transport negotiation, which occurs up front, needs to progress differently than port 80.
I don't think there is any practical reason why the protocol/scheme name can't be used to select a driver/stream-handler like software implementation. The protocol handler will have the scheme of the URL will it not?
Gregg Wonderly
On Jan 7, 2013, at 4:42 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
> On 7/01/2013 11:32 AM, Gregg Wonderly wrote:
>> I think that using "reference" rather than a protocol string is not a good idea, because it puts this important differentiation at the end rather than first. Just using
>>
>> jini: for discovery V1 or V2
>> usb: for something over a USB interface of yet to be decided nature
>> udt: for UDT
>> mekong: for mekong
>>
>> etc., would be the right choice. For me, that would provide a better bases for a protocol handler/driver mechanism.
>>
>> Gregg Wonderly
>
> I did consider that, but it clobbers dns, we loose the scheme information, making life harder if we wanted to do dns-srv discovery. (Not that you'll use dns-srv for usb, but you might for mekong or udt).
>
> Someone using dns srv to discover udt won't know whether it's jini or something else.
>
> You're right though, using the fragment or reference component is a hack.
>
> We could use the path segment, it has no meaning presently for discovery, jini://.
>
> "/" could potentially be used as a separator for nested Socket providers.
>
> This might allow the client and server to nest Sockets in an identical hierarchy.
>
> discovery SSL
> |
> DeflateSocket (compression)
> |
> MekongSocket
>
> Cheers,
>
> Peter.
>
Re: Discovery v3
Posted by Peter Firmstone <ji...@zeus.net.au>.
On 7/01/2013 11:32 AM, Gregg Wonderly wrote:
> I think that using "reference" rather than a protocol string is not a good idea, because it puts this important differentiation at the end rather than first. Just using
>
> jini: for discovery V1 or V2
> usb: for something over a USB interface of yet to be decided nature
> udt: for UDT
> mekong: for mekong
>
> etc., would be the right choice. For me, that would provide a better bases for a protocol handler/driver mechanism.
>
> Gregg Wonderly
I did consider that, but it clobbers dns, we loose the scheme
information, making life harder if we wanted to do dns-srv discovery.
(Not that you'll use dns-srv for usb, but you might for mekong or udt).
Someone using dns srv to discover udt won't know whether it's jini or
something else.
You're right though, using the fragment or reference component is a hack.
We could use the path segment, it has no meaning presently for
discovery, jini://.
"/" could potentially be used as a separator for nested Socket providers.
This might allow the client and server to nest Sockets in an identical
hierarchy.
discovery SSL
|
DeflateSocket (compression)
|
MekongSocket
Cheers,
Peter.
Re: Discovery v3
Posted by Gregg Wonderly <gr...@wonderly.org>.
I think that using "reference" rather than a protocol string is not a good idea, because it puts this important differentiation at the end rather than first. Just using
jini: for discovery V1 or V2
usb: for something over a USB interface of yet to be decided nature
udt: for UDT
mekong: for mekong
etc., would be the right choice. For me, that would provide a better bases for a protocol handler/driver mechanism.
Gregg Wonderly
On Jan 5, 2013, at 3:03 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
> Gregg,
>
> Now you've got me thinking about Discovery v3 ;)
>
> replace hostname and port with a RFC3986 uri string.
>
> expand the URI format to allow other sockets, for example:
>
> * jini://hostname:port#IP - TCP/IP Socket
> * jini://hostid#USB- USB Socket
> * jini://hostname:port?socID=500#UDT - UDT Socket with socketID for
> multiplexing 10x faster than TCP sockets and firewall friendly.
> * jini://hostname:port#MEKONG - Sim?
>
>
> replace SHA-1 hash functions with SHA-2 in |net.jini.discovery.ssl| format.
>
> Cheers,
>
> Peter.
>
>