You are viewing a plain text version of this content. The canonical link for it is here.
Posted to xindice-dev@xml.apache.org by James Bates <ja...@amplexor.com> on 2002/03/26 11:49:11 UTC
open issues
OK guys; sorry for having not responded earlier... I've been kind of busy (at the OpenCMS conference in Zürich; Xindice was mentioned 3 times! Yippie...) and will be busy for a few more days on some non-Xindice stuff... However after that, I fully intend once again to participate actively in the project.
Some points I'd like to mention:
- about the internal RPC API:
====================
Kimbro you proposed a solution passing big hashtables back and forth accross the RPC tool. I'm sorry, but I don't see the point? At the bear-bones level, you'll always need server code that implements the various methods/functions/operations/whatever-you-call-them with the appropriate arguments that you need. If you want to add a new method/function/whatever, you'll still have to add server code...
Equally the client needs to know the available methods and still needs to prepare parameters and send them, using some means for each function/operation/method it needs.
Now in a pure RPC approach, client and server would do their stuff (implementing functions (server) and calling functions (client) using constructus familiar to that language itself (Java, C, perl, whatever) and the RPC tool should take care of seeing everything gets sent accross the network as it should. Also we agreed that XML-RPC does this well, even intl chars, in the special case of Java clients and servers (which is what we do in the internal API).
If you make big hashtables and write only one Java method on the server that checks the hashtable and looks up a method, casts the hashtable's contents and sends them to the appropriate method, aren't you actually duplicating the server-side skeleton functionality? something that the RPC toolkit already could do (and does) really well? I don't see the gain, because you still need to write all server functionality, and you need to support an additional server marshalling/unmarshalling/dispatching function...
I'm fine by XML-RPC or whatever RPC tool, but I think we should really USE it.
- Using SOAP:
==========
As long as all unicode characters can be handled when passing java.lang.String's accross XML-RPC from JAVA code to JAVA code, I'm fine with any RPC tool, and I agree XML-RPC is simpler, so would prefer it. (I simply didn't know about the setEncoding() before). I also fully understand that non-unicode languages can't be expected to handle these incoming strings naturally in their own string equivalent, and even with SOAP this wouldn't improve anyway. Besides we don't need other languages in developing the internal API.
Any external API could provide alternatives sending byte-streams or base64 strings or whatever.
In light of all this, I support the view of most people here I think, which is to have a SOAP interface as a "high-level" API, alongside possible a high-level XML-RPC, a WebDAV interface, and possibly others
all built on the internal API...
- Patching code to use UTF-8 as internal string representation:
============================================
Indeed I will start patching the code to do this, in a few days. I repeat that the major outstanding issue here, is that Strings passed accross the CORBA API weren't handled well, causing problems for all queries, amongst other things. Together with the switch to RPC, this should be resolved automatically.
But I repeat my commitment to fix this probably by next week.
Also if my reasoing in the first points convinces anyone, I'd like to write a XML:DB connector that behaves as a client to the original well-defined RPC JAVA API I had made, to complete the cycle and get something fully operational.
James
Re: open issues
Posted by Paul Hammant <ha...@apache.org>.
James, Folks,
In case you did not know there is a Java only RPC tool called AltRMI.
It is going to be faster than going to XML etc, and if you had a need
to use it for this project I'd be more than happy to help. It, of
course, breaks one of your design goals (to use XML for
interoperability), but if your need is /entirely/ internal to Xindice
even in a federation of Xindice servers, I don't see a problem. Horses
for courses - it is fast and has multiple transports, has a decent
interface/impl separation.
See http://cvs.apache.org/viewcvs/jakarta-commons-sandbox/altrmi/
Some of the classes in the test package should be self explanatory.
It is in use in Avalon (generic "transport" package in
avalon-cornerstone and client for new "instrument" package in
avalon-excalibur). Two of the demos in avalon-apps use the transport
package. An entire off Apache project called Enterprise Object Broker
(EOB) is using it and a few private companies are using it for unknown
tasks.
It has its uses and limitations, but you'd be fooish to dismiss it after
a one minute analysis.
- Paul
> OK guys; sorry for having not responded earlier... I've been kind of
> busy (at the OpenCMS conference in Zürich; Xindice was mentioned 3
> times! Yippie...) and will be busy for a few more days on some
> non-Xindice stuff... However after that, I fully intend once again to
> participate actively in the project.
>
> Some points I'd like to mention:
>
> - about the internal RPC API:
> ====================
> Kimbro you proposed a solution passing big hashtables back and forth
> accross the RPC tool. I'm sorry, but I don't see the point? At the
> bear-bones level, you'll always need server code that implements the
> various methods/functions/operations/whatever-you-call-them with the
> appropriate arguments that you need. If you want to add a new
> method/function/whatever, you'll still have to add server code...
>
> Equally the client needs to know the available methods and still needs
> to prepare parameters and send them, using some means for each
> function/operation/method it needs.
>
> Now in a pure RPC approach, client and server would do their stuff
> (implementing functions (server) and calling functions (client) using
> constructus familiar to that language itself (Java, C, perl, whatever)
> and the RPC tool should take care of seeing everything gets sent
> accross the network as it should. Also we agreed that XML-RPC does
> this well, even intl chars, in the special case of Java clients and
> servers (which is what we do in the internal API).
>
> If you make big hashtables and write only one Java method on the
> server that checks the hashtable and looks up a method, casts the
> hashtable's contents and sends them to the appropriate method, aren't
> you actually duplicating the server-side skeleton functionality?
> something that the RPC toolkit already could do (and does) really
> well? I don't see the gain, because you still need to write all server
> functionality, and you need to support an additional server
> marshalling/unmarshalling/dispatching function...
>
> I'm fine by XML-RPC or whatever RPC tool, but I think we should really
> USE it.
>
> - Using SOAP:
> ==========
> As long as all unicode characters can be handled when passing
> java.lang.String's accross XML-RPC from JAVA code to JAVA code, I'm
> fine with any RPC tool, and I agree XML-RPC is simpler, so would
> prefer it. (I simply didn't know about the setEncoding() before). I
> also fully understand that non-unicode languages can't be expected to
> handle these incoming strings naturally in their own string
> equivalent, and even with SOAP this wouldn't improve anyway. Besides
> we don't need other languages in developing the internal API.
>
> Any external API could provide alternatives sending byte-streams or
> base64 strings or whatever.
>
> In light of all this, I support the view of most people here I think,
> which is to have a SOAP interface as a "high-level" API, alongside
> possible a high-level XML-RPC, a WebDAV interface, and possibly others
>
> all built on the internal API...
>
> - Patching code to use UTF-8 as internal string representation:
> ============================================
> Indeed I will start patching the code to do this, in a few days. I
> repeat that the major outstanding issue here, is that Strings passed
> accross the CORBA API weren't handled well, causing problems for all
> queries, amongst other things. Together with the switch to RPC, this
> should be resolved automatically.
>
> But I repeat my commitment to fix this probably by next week.
>
> Also if my reasoing in the first points convinces anyone, I'd like to
> write a XML:DB connector that behaves as a client to the original
> well-defined RPC JAVA API I had made, to complete the cycle and get
> something fully operational.
>
>
> James
>
>
Re: open issues
Posted by Kurt Ward <ku...@yahoo.com>.
> > OK guys; sorry for having not responded earlier... I've been kind of
busy
> > (at the OpenCMS conference in Zürich; Xindice was mentioned 3 times!
> > Yippie...) and will be busy for a few more days on some non-Xindice
stuff.
> > .. However after that, I fully intend once again to participate actively
> > in the project.
> >
> > Some points I'd like to mention:
> >
> > - about the internal RPC API:
> > ====================
> > Kimbro you proposed a solution passing big hashtables back and forth
> > accross the RPC tool. I'm sorry, but I don't see the point? At the
> > bear-bones level, you'll always need server code that implements the
> > various methods/functions/operations/whatever-you-call-them with the
> > appropriate arguments that you need. If you want to add a new
> > method/function/whatever, you'll still have to add server code...
>
> Ahh, OK. Let me explain further, there is a point and I still believe it's
> a worthy one. I was hoping the experimental implementation would be enough
> to make it clear where the potential lies. This is one of those times I
> wish we could just get together in front of a whiteboard and hash it out.
>
> The problem is simply that adding parameters to existing method calls
> changes the signature of those method calls. This automatically breaks all
> existing clients even in cases where those clients may not need to have
> been broken. Basically my goal is for the API to be easy to change. We
> need this because there is so much about what we're doing that is still
> really fuzzy.
>
> Let's take an example of transactions. The current API as designed does
> not support transactions. So how do we add transaction support? Well for
> one thing we're probably going to have to pass around a transaction ID of
> some type and this ID will need to be passed to any method that operates
> within the context of that transaction. This change to the existing API
> will break all current clients because the method signatures will have to
> change. This is unnecessary breakage though, as those clients could have
> continued to work with an implicit transaction model. The transaction ID
> is a completely optional parameter, but the traditional RPC model forces
> you to deal with it even if you don't want to.
>
> Another example is a method to configure some object, let's say indexes.
> What happens when we add a new index type like full text indexing that may
> require different parameters to configure it? In a standard RPC API you
> break all clients that call that method and force them to deal with the
> fact that you added those parameters. When in reality, they could have
> continued to work in blissful contentment, ignorant of the fact that full
> text even existed in the server.
>
> Really this is a problem because we're using Java and the XML-RPC
> implementation uses both the method name and its signature to find the
> method to call. XML-RPC was originally designed to be used with scripting
> languages and it usually only uses the method name to locate the code to
> call. This allows the parameter list to be fuzzy. In the Apache XML-RPC
> impl you can achieve this by implementing XmlRpcHandler, however you're
> still relying on parameter order to resolve things.
>
> You of course have to add server code to add new functionality, however if
> you look at the implementation what you're adding is a class not a method
> to a class. You can add classes at any time very easily. Also to add a
> parameter to a message you simply change the internals of the class not
> the public signature. This makes the API much more malleable without
> automatically breaking clients.
>
> Yes there will be cases where you want to add a parameter and make it
> required, but that is a simple check that should generate an informative
> error message from the class.
Thanks for the explanation on this. I now see what you are trying to
achieve
and it does make a lot of sense.
> Also I really want to replace XMLObjects with user definable API messages.
> It's a massively simpler model then the heavy introspection based model
> of XMLObjects and will be much easier to call over the wire.
I like the idea of replacing XMLObjects with something a little easier to
use as long
as the replacement provides the same flexibility. I've started using them
(XMLObjects)
more and have really grown to like them. What do you have in mind?
Kurt
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
Re: open issues
Posted by Kimbro Staken <ks...@xmldatabases.org>.
On Tuesday, March 26, 2002, at 03:49 AM, James Bates wrote:
> OK guys; sorry for having not responded earlier... I've been kind of busy
> (at the OpenCMS conference in Zürich; Xindice was mentioned 3 times!
> Yippie...) and will be busy for a few more days on some non-Xindice stuff.
> .. However after that, I fully intend once again to participate actively
> in the project.
>
> Some points I'd like to mention:
>
> - about the internal RPC API:
> ====================
> Kimbro you proposed a solution passing big hashtables back and forth
> accross the RPC tool. I'm sorry, but I don't see the point? At the
> bear-bones level, you'll always need server code that implements the
> various methods/functions/operations/whatever-you-call-them with the
> appropriate arguments that you need. If you want to add a new
> method/function/whatever, you'll still have to add server code...
Ahh, OK. Let me explain further, there is a point and I still believe it's
a worthy one. I was hoping the experimental implementation would be enough
to make it clear where the potential lies. This is one of those times I
wish we could just get together in front of a whiteboard and hash it out.
The problem is simply that adding parameters to existing method calls
changes the signature of those method calls. This automatically breaks all
existing clients even in cases where those clients may not need to have
been broken. Basically my goal is for the API to be easy to change. We
need this because there is so much about what we're doing that is still
really fuzzy.
Let's take an example of transactions. The current API as designed does
not support transactions. So how do we add transaction support? Well for
one thing we're probably going to have to pass around a transaction ID of
some type and this ID will need to be passed to any method that operates
within the context of that transaction. This change to the existing API
will break all current clients because the method signatures will have to
change. This is unnecessary breakage though, as those clients could have
continued to work with an implicit transaction model. The transaction ID
is a completely optional parameter, but the traditional RPC model forces
you to deal with it even if you don't want to.
Another example is a method to configure some object, let's say indexes.
What happens when we add a new index type like full text indexing that may
require different parameters to configure it? In a standard RPC API you
break all clients that call that method and force them to deal with the
fact that you added those parameters. When in reality, they could have
continued to work in blissful contentment, ignorant of the fact that full
text even existed in the server.
Really this is a problem because we're using Java and the XML-RPC
implementation uses both the method name and its signature to find the
method to call. XML-RPC was originally designed to be used with scripting
languages and it usually only uses the method name to locate the code to
call. This allows the parameter list to be fuzzy. In the Apache XML-RPC
impl you can achieve this by implementing XmlRpcHandler, however you're
still relying on parameter order to resolve things.
You of course have to add server code to add new functionality, however if
you look at the implementation what you're adding is a class not a method
to a class. You can add classes at any time very easily. Also to add a
parameter to a message you simply change the internals of the class not
the public signature. This makes the API much more malleable without
automatically breaking clients.
Yes there will be cases where you want to add a parameter and make it
required, but that is a simple check that should generate an informative
error message from the class.
Also I really want to replace XMLObjects with user definable API messages.
It's a massively simpler model then the heavy introspection based model
of XMLObjects and will be much easier to call over the wire.
>
> Equally the client needs to know the available methods and still needs to
> prepare parameters and send them, using some means for each
> function/operation/method it needs.
>
Changing the server API will always require changes to the client code to
take advantage of the new functionality. What we want to avoid is
unnecessarily breaking clients because we added some functionality that
they don't care about.
> Now in a pure RPC approach, client and server would do their stuff
> (implementing functions (server) and calling functions (client) using
> constructus familiar to that language itself (Java, C, perl, whatever)
> and the RPC tool should take care of seeing everything gets sent accross
> the network as it should. Also we agreed that XML-RPC does this well,
> even intl chars, in the special case of Java clients and servers (which
> is what we do in the internal API).
You're still using it for this purpose. Unfortunately pure RPC based APIs
are brittle and this was a massive problem with the CORBA API. I'm really
hoping we can not repeat that sin. The message passing is still XML-RPC,
it still uses the serialization of the library and the server
implementation is still fairly simple.
>
> If you make big hashtables and write only one Java method on the server
> that checks the hashtable and looks up a method, casts the hashtable's
> contents and sends them to the appropriate method, aren't you actually
> duplicating the server-side skeleton functionality? something that the
> RPC toolkit already could do (and does) really well? I don't see the gain,
> because you still need to write all server functionality, and you need
> to support an additional server marshalling/unmarshalling/dispatching
> function...
All you do is extract the method name, find the class that handles the
message and hand the whole Hashtable to that class. It's extremely simple.
It's entirely up to the loaded class to interpret the rest of the message
contents but the XML-RPC library still handles marshaling.
> Also if my reasoing in the first points convinces anyone, I'd like to
> write a XML:DB connector that behaves as a client to the original
> well-defined RPC JAVA API I had made, to complete the cycle and get
> something fully operational.
>
Sorry for trying to change this up and introducing more complexity. Don't
worry though, I'll provide a full implementation of the message oriented
API shortly including XML:DB support. After working with it a bit, I'm
firmly convinced that it is a better way. Most of the server pieces are
already done, but I want to build some good unit tests and then I'll
convert the XML:DB API.
>
> James
>
>
>
Kimbro Staken
Java and XML Software, Consulting and Writing http://www.xmldatabases.org/
Apache Xindice native XML database http://xml.apache.org/xindice
XML:DB Initiative http://www.xmldb.org