You are viewing a plain text version of this content. The canonical link for it is here.
Posted to soap-dev@xml.apache.org by Nathan Wray <nw...@mich.com> on 2000/10/05 11:40:26 UTC
SSL and Soap
George;
It's a change to the original codebase that incorporates your Properties file loading and proxy support, and works with HTTP or HTTPS. I'm planning to change it further by removing the in-line string compares and replacing it with class-scope booleans that are set once with static system property checks for performance.
I'm concerned that the codebase you're proposing will have a lot of unnecessary overhead per-call. I'm not sure of your motivation in rebuilding the URL object on every call, it seems to be saying that you don't trust the developer to create the URL object they want (and incurring an extra object creation per-call). I intend to have the client side code create a URL object on initialization and use it forever, and having the code re-create it internally for each connection defeats that. Similarly, trying to throw an initialization call into every entry point is wasteful and ultimately you'll miss one. Why not make the call part of the class load with a static block?
I might also rename the HTTPUtils class to URLUtils, since it should work with any URL object and it's unnecessarily restrictive to call it HTTP utils.
I'll be taking this code into production within the next few months and it needs to be able to handle hundreds of thousands of calls a day peaking at 100+ calls a second, so every extra call, object creation and string concat costs.
Respectfully yours
Nathan
George I Matkovits wrote:
> I like changes to changes which simplify the changes and will also help to accomplish the next step: Server Access Control ASAP.
> Regards - George
>
> Nathan Wray wrote:
>
> > George, what if I send you another version that accomplishes the same results with fewer changes?
> >
> > George I Matkovits wrote:
> >
> > > Hello Nathan,
> > > Thank you for the comments and just a quick reply. I will do more thinking later off line.
> > > 1.) Multiple calls: just in case I missed an entry point. The 'problem' with object programming is that it can be put together in many, many ways and I would like to have the properties defined ASAP. Eventually I would like to use them even for code compatibility 'on the fly' reconfiguration, like 'MS compatibility' or 'Standard Soap Compatibility' depending on what server you want to talk to from your Java Client.
> > > 2.) I am using URL and not Socket connections because SSL Sockets (or proxies) do not seem to work with Sun's SSL1.0.1 release. This is why both HTTPUtils and HTTPSUtils are required. HTTPUtils is used with an http endpoint and HTTPSUtils with an https endpoint, this way we are able to stay 'binary' compatible with the current Apache Soap and need only 2 lines of additional code in the samples (or any existing client code) to make them work over https. An https server is rather complicated to set up and
> > > must 'newbie' Soap user will never work in a commercial environment where https is the only way to go. They just want to play and learn Soap with the binary release. They will not need the extra jars from Sun. Most members of the xml-user list have problems even with installing Tomcat and the few jars. IMHO when they become familiar with the http version then by adding the 3 Sun jars they can easily 'graduate' to the Secure Version. There is going to be a very slight increase in overhead but the flexibility
> > > of backward compatibility, easy upgrade from the 'training wheels' version to the fully secure 'heavy duty' version is IMHO probably worth it (I am sure you guessed by now that I am and 'automate it' lunatic. One must be to be able to compete with M$$ :-)
> > > 3.) re creating the URL just makes me feel comfortable, just in case the incoming URL is miss formed. I have no real strong opinion on this.
> > > Any more comments (even violent ones :-) would be much appreciated...... I am trying to organize with my boss' help a code review of this ASAP and I will try to represent your views and let the reviewers see your EMails.
> > > Regards - George
> > >
> > > Nathan Wray wrote:
> > >
> > > > Hello George;
> > > >
> > > > I've taken a high level look at the new code. I'll get right to the issues as I see them:
> > > >
> > > > I'm not sure it's necessary to add 10 in-line calls to the "PropertiesUtils.IniPropertiesUtils()" method to insure it's called. You could pick one key class and add it as static code to the class so it's only called once on initialization rather than several times per SOAP call. I realize it's a do-nothing call after the first one but there is unnecessary per-call overhead to build the call stack and return.
> > > >
> > > > I think the SOAP code could be done w/o a HTTPSUtils class, and without rebuilding the URL object. I don't see the advantage of the more complicated code and it will certianly hurt performance.
> > > >
> > > > In short, I'd rather see a few lines in the origional codebase change to add proxy support and to allow an https URL to work correctly, as well as a static SOAP properties loader, and do away with the URL rebuilding and in-line init calls.
> > > >
> > > > thanks for listening.
> > > >
> > > > George I Matkovits wrote:
> > > >
> > > > > Here is the last part. After I finished with the Server Security Additions I will start working with MS-Java compatibility issues. IMHO we should keep Java Soap 'dynamically configurable' with the help of Soap.properties to comply either with the 'open standard' or the de-facto MS standard. MS will keep on changing its implementation (1) to fix its bugs (2) to keep its 'platform' competitor's off balance. I live in a mixed platform environment and we must be able to keep MS and the 'standard' version
> > > > > in sink. I am also planning to develop a generic WSTK interface generator for our applications (around 2000-3000 apps) which are currently all web enabled. Please look at the 2 additional 'Secure' samples. Life would have been much easier if the URL class could be sub classed but it is a 'final' class currently.
> > > > > Regards - George
> > > > > p.s. I start entering this into the Apache CVS but will not commit the changes until your reply comes back. Than you.
> >
> > --
> > Nathan Wray
> > nwray@mich.com
> > --
> > "Can anybody cite a single interesting or
> > important idea or argument that's emerged
> > from the months of campaigning in the
> > current U.S. presidential race?"
> > Jon Katz, slashdot.org
> > http://slashdot.org/article.pl?sid=00/09/19/035231&mode=nested
--
Nathan Wray
nwray@mich.com
--
"Can anybody cite a single interesting or
important idea or argument that's emerged
from the months of campaigning in the
current U.S. presidential race?"
Jon Katz, slashdot.org
http://slashdot.org/article.pl?sid=00/09/19/035231&mode=nested
Re: SSL and Soap + Secure Soap (access validation and credentials
generation/checking)
Posted by George I Matkovits <ma...@uswest.net>.
We must also 'fix' the current security holes ASAP without too much impact on performance and usage complexity. That is what I am trying to do. V3.0 Soap will be nirvana (-:, eventually.
Than you for the help.
Regards - George
p.s. I run into a 'timing' problem with Soap.properties initialization within the class constructor but a static block (at least on my machine) based initializer seems to have solved it. I will post and send the latest version (with some of the payload signing capability in the code) to you later this week. The Soap 'wire protocol' is going to be used over the open Internet and without SSL or authentication it WILL BE USELESS. I believe that we have it now as simple as can be. I also run into overhead problems with payload signing and have a 'simplified' approach working already. I will post it ALL again after some more testing. The simplification is briefly:
1.) I generate my own PKI keys with a new java utility (will be part of the release). Now there is no need for a Client Certificate (could be upto $600/year, or only 20 days, for a 'real' test certificate).
2.) Add only the client's Distinguished Name (as a header), both signed and 'in clear', to the Payload. Now the SSL Server is able to authenticate from the signature and there is no 'full payload' signing overhead ( a very big deal for performance!!!!). The client DN signature is also done by the above utility. Many thanks to all the reviewers of SecureSoap to make me simplify, simplify, ....!
Regards - George
Nathan Wray wrote:
> We should consider carefully which components belong in a wire protocol such as SOAP, and which components belong in a higher level of abstraction. It may be useful to create a toolset on top of SOAP without burdoning the underlying libraries with unnecessary restrictions on the developer.
>
> George I Matkovits wrote:
>
> > I am going to try and incorporate any improvements. I am also on my way with the Security enhancements. They both will become part of of the 2.0 source tree when testing is completed. I would be also very happy with any additional suggestions, extensions, improvements, testing (this is OPEN SOURCE! :-). The Security DB will be a flat file like we are using for the Server's Service classes, for efficiency and ease of installation. I gave up on the SQL idea when realized its installation and maintenance problems. Most people on the soap-user forum (or I on Linux) could not manage to install mySQL and JDBC/ODBC drivers without difficulty. I managed to generate my own PKI (128
> > byte) keys on NT yesterday (for both RSA and DHA) with the Java2/V1.3 APIs. (Now I must also upgrade my Linux System to Sun's V1.3!) This way we will not need Client 'test certificates'. Any production runs will eventually need it. The PKI Generator (Public/Private keys) is just an 'off-line' utility. It takes less then a minute on my 366 MHz Windows2000 System. Verisign is going to kill me. (-:
> > Regards - George
> > p.s most of the 'over head' is connection time and now with 'keep alive' operating against Apache or IIS this is minimal for repeated (even SSL) calls. But all and every overhead reduction will count for scale-ability! I will change the the class load to do Properties initialization and creation as you suggested. A great idea, I should have done it that way from the beginning. It would have also reduced test time. I am rebuilding the URL to prevent any possible client screw ups like we had with the 'infamous' Port# bug. I will try and do some kind of 'reuse' also. IMHO it is not rebuilt, even now, when you are not using the AutoSSLEnable but just the SSLEnable request. (Or
> > something like that, the code is on a different system then my EMAIL.)
> >
>
> --
> Nathan Wray
> nwray@mich.com
> --
> "I'll keep it short and sweet.
> Family. Religion. Friendship.
> These are the three demons you must slay if
> you wish to succeed in business."
>
> - Charles Montgomery Burns
> (from The Simpsons)
Re: SSL and Soap + Secure Soap (access validation and credentials
generation/checking)
Posted by George I Matkovits <ma...@uswest.net>.
We must also 'fix' the current security holes ASAP without too much impact on performance and usage complexity. That is what I am trying to do. V3.0 Soap will be nirvana (-:, eventually.
Than you for the help.
Regards - George
p.s. I run into a 'timing' problem with Soap.properties initialization within the class constructor but a static block (at least on my machine) based initializer seems to have solved it. I will post and send the latest version (with some of the payload signing capability in the code) to you later this week. The Soap 'wire protocol' is going to be used over the open Internet and without SSL or authentication it WILL BE USELESS. I believe that we have it now as simple as can be. I also run into overhead problems with payload signing and have a 'simplified' approach working already. I will post it ALL again after some more testing. The simplification is briefly:
1.) I generate my own PKI keys with a new java utility (will be part of the release). Now there is no need for a Client Certificate (could be upto $600/year, or only 20 days, for a 'real' test certificate).
2.) Add only the client's Distinguished Name (as a header), both signed and 'in clear', to the Payload. Now the SSL Server is able to authenticate from the signature and there is no 'full payload' signing overhead ( a very big deal for performance!!!!). The client DN signature is also done by the above utility. Many thanks to all the reviewers of SecureSoap to make me simplify, simplify, ....!
Regards - George
Nathan Wray wrote:
> We should consider carefully which components belong in a wire protocol such as SOAP, and which components belong in a higher level of abstraction. It may be useful to create a toolset on top of SOAP without burdoning the underlying libraries with unnecessary restrictions on the developer.
>
> George I Matkovits wrote:
>
> > I am going to try and incorporate any improvements. I am also on my way with the Security enhancements. They both will become part of of the 2.0 source tree when testing is completed. I would be also very happy with any additional suggestions, extensions, improvements, testing (this is OPEN SOURCE! :-). The Security DB will be a flat file like we are using for the Server's Service classes, for efficiency and ease of installation. I gave up on the SQL idea when realized its installation and maintenance problems. Most people on the soap-user forum (or I on Linux) could not manage to install mySQL and JDBC/ODBC drivers without difficulty. I managed to generate my own PKI (128
> > byte) keys on NT yesterday (for both RSA and DHA) with the Java2/V1.3 APIs. (Now I must also upgrade my Linux System to Sun's V1.3!) This way we will not need Client 'test certificates'. Any production runs will eventually need it. The PKI Generator (Public/Private keys) is just an 'off-line' utility. It takes less then a minute on my 366 MHz Windows2000 System. Verisign is going to kill me. (-:
> > Regards - George
> > p.s most of the 'over head' is connection time and now with 'keep alive' operating against Apache or IIS this is minimal for repeated (even SSL) calls. But all and every overhead reduction will count for scale-ability! I will change the the class load to do Properties initialization and creation as you suggested. A great idea, I should have done it that way from the beginning. It would have also reduced test time. I am rebuilding the URL to prevent any possible client screw ups like we had with the 'infamous' Port# bug. I will try and do some kind of 'reuse' also. IMHO it is not rebuilt, even now, when you are not using the AutoSSLEnable but just the SSLEnable request. (Or
> > something like that, the code is on a different system then my EMAIL.)
> >
>
> --
> Nathan Wray
> nwray@mich.com
> --
> "I'll keep it short and sweet.
> Family. Religion. Friendship.
> These are the three demons you must slay if
> you wish to succeed in business."
>
> - Charles Montgomery Burns
> (from The Simpsons)
Re: SSL and Soap + Secure Soap (access validation and credentials
generation/checking)
Posted by Nathan Wray <nw...@mich.com>.
We should consider carefully which components belong in a wire protocol such as SOAP, and which components belong in a higher level of abstraction. It may be useful to create a toolset on top of SOAP without burdoning the underlying libraries with unnecessary restrictions on the developer.
George I Matkovits wrote:
> I am going to try and incorporate any improvements. I am also on my way with the Security enhancements. They both will become part of of the 2.0 source tree when testing is completed. I would be also very happy with any additional suggestions, extensions, improvements, testing (this is OPEN SOURCE! :-). The Security DB will be a flat file like we are using for the Server's Service classes, for efficiency and ease of installation. I gave up on the SQL idea when realized its installation and maintenance problems. Most people on the soap-user forum (or I on Linux) could not manage to install mySQL and JDBC/ODBC drivers without difficulty. I managed to generate my own PKI (128
> byte) keys on NT yesterday (for both RSA and DHA) with the Java2/V1.3 APIs. (Now I must also upgrade my Linux System to Sun's V1.3!) This way we will not need Client 'test certificates'. Any production runs will eventually need it. The PKI Generator (Public/Private keys) is just an 'off-line' utility. It takes less then a minute on my 366 MHz Windows2000 System. Verisign is going to kill me. (-:
> Regards - George
> p.s most of the 'over head' is connection time and now with 'keep alive' operating against Apache or IIS this is minimal for repeated (even SSL) calls. But all and every overhead reduction will count for scale-ability! I will change the the class load to do Properties initialization and creation as you suggested. A great idea, I should have done it that way from the beginning. It would have also reduced test time. I am rebuilding the URL to prevent any possible client screw ups like we had with the 'infamous' Port# bug. I will try and do some kind of 'reuse' also. IMHO it is not rebuilt, even now, when you are not using the AutoSSLEnable but just the SSLEnable request. (Or
> something like that, the code is on a different system then my EMAIL.)
>
--
Nathan Wray
nwray@mich.com
--
"I'll keep it short and sweet.
Family. Religion. Friendship.
These are the three demons you must slay if
you wish to succeed in business."
- Charles Montgomery Burns
(from The Simpsons)
Re: SSL and Soap + Secure Soap (access validation and credentials
generation/checking)
Posted by Nathan Wray <nw...@mich.com>.
We should consider carefully which components belong in a wire protocol such as SOAP, and which components belong in a higher level of abstraction. It may be useful to create a toolset on top of SOAP without burdoning the underlying libraries with unnecessary restrictions on the developer.
George I Matkovits wrote:
> I am going to try and incorporate any improvements. I am also on my way with the Security enhancements. They both will become part of of the 2.0 source tree when testing is completed. I would be also very happy with any additional suggestions, extensions, improvements, testing (this is OPEN SOURCE! :-). The Security DB will be a flat file like we are using for the Server's Service classes, for efficiency and ease of installation. I gave up on the SQL idea when realized its installation and maintenance problems. Most people on the soap-user forum (or I on Linux) could not manage to install mySQL and JDBC/ODBC drivers without difficulty. I managed to generate my own PKI (128
> byte) keys on NT yesterday (for both RSA and DHA) with the Java2/V1.3 APIs. (Now I must also upgrade my Linux System to Sun's V1.3!) This way we will not need Client 'test certificates'. Any production runs will eventually need it. The PKI Generator (Public/Private keys) is just an 'off-line' utility. It takes less then a minute on my 366 MHz Windows2000 System. Verisign is going to kill me. (-:
> Regards - George
> p.s most of the 'over head' is connection time and now with 'keep alive' operating against Apache or IIS this is minimal for repeated (even SSL) calls. But all and every overhead reduction will count for scale-ability! I will change the the class load to do Properties initialization and creation as you suggested. A great idea, I should have done it that way from the beginning. It would have also reduced test time. I am rebuilding the URL to prevent any possible client screw ups like we had with the 'infamous' Port# bug. I will try and do some kind of 'reuse' also. IMHO it is not rebuilt, even now, when you are not using the AutoSSLEnable but just the SSLEnable request. (Or
> something like that, the code is on a different system then my EMAIL.)
>
--
Nathan Wray
nwray@mich.com
--
"I'll keep it short and sweet.
Family. Religion. Friendship.
These are the three demons you must slay if
you wish to succeed in business."
- Charles Montgomery Burns
(from The Simpsons)
Re: SSL and Soap + Secure Soap (access validation and credentials
generation/checking)
Posted by George I Matkovits <ma...@uswest.net>.
I am going to try and incorporate any improvements. I am also on my way with the Security enhancements. They both will become part of of the 2.0 source tree when testing is completed. I would be also very happy with any additional suggestions, extensions, improvements, testing (this is OPEN SOURCE! :-). The Security DB will be a flat file like we are using for the Server's Service classes, for efficiency and ease of installation. I gave up on the SQL idea when realized its installation and maintenance problems. Most people on the soap-user forum (or I on Linux) could not manage to install mySQL and JDBC/ODBC drivers without difficulty. I managed to generate my own PKI (128
byte) keys on NT yesterday (for both RSA and DHA) with the Java2/V1.3 APIs. (Now I must also upgrade my Linux System to Sun's V1.3!) This way we will not need Client 'test certificates'. Any production runs will eventually need it. The PKI Generator (Public/Private keys) is just an 'off-line' utility. It takes less then a minute on my 366 MHz Windows2000 System. Verisign is going to kill me. (-:
Regards - George
p.s most of the 'over head' is connection time and now with 'keep alive' operating against Apache or IIS this is minimal for repeated (even SSL) calls. But all and every overhead reduction will count for scale-ability! I will change the the class load to do Properties initialization and creation as you suggested. A great idea, I should have done it that way from the beginning. It would have also reduced test time. I am rebuilding the URL to prevent any possible client screw ups like we had with the 'infamous' Port# bug. I will try and do some kind of 'reuse' also. IMHO it is not rebuilt, even now, when you are not using the AutoSSLEnable but just the SSLEnable request. (Or
something like that, the code is on a different system then my EMAIL.)
Rich Johns wrote:
> Is all this stuff going to become part of the 2.0 source tree? I've lost track of
> what version has what functionality.
>
> thanks.
>
> Nathan Wray wrote:
>
> >
> > George;
> >
> > It's a change to the original codebase that incorporates your Properties file loading and proxy support, and works with HTTP or HTTPS. I'm planning to change it further by removing the in-line string compares and replacing it with class-scope booleans that are set once with static system property checks for performance.
> >
> > I'm concerned that the codebase you're proposing will have a lot of unnecessary overhead per-call. I'm not sure of your motivation in rebuilding the URL object on every call, it seems to be saying that you don't trust the developer to create the URL object they want (and incurring an extra object creation per-call). I intend to have the client side code create a URL object on initialization and use it forever, and having the code re-create it internally for each connection defeats that. Similarly, trying to throw an initialization call into every entry point is wasteful and ultimately you'll miss one. Why not make the call part of the class load with a static block?
> >
> > I might also rename the HTTPUtils class to URLUtils, since it should work with any URL object and it's unnecessarily restrictive to call it HTTP utils.
> >
> > I'll be taking this code into production within the next few months and it needs to be able to handle hundreds of thousands of calls a day peaking at 100+ calls a second, so every extra call, object creation and string concat costs.
> >
> > Respectfully yours
> > Nathan
> >
> >
> > George I Matkovits wrote:
> >
> >> I like changes to changes which simplify the changes and will also help to accomplish the next step: Server Access Control ASAP.
> >> Regards - George
> >>
> >> Nathan Wray wrote:
> >>
> >> > George, what if I send you another version that accomplishes the same results with fewer changes?
> >> >
> >> > George I Matkovits wrote:
> >> >
> >> > > Hello Nathan,
> >> > > Thank you for the comments and just a quick reply. I will do more thinking later off line.
> >> > > 1.) Multiple calls: just in case I missed an entry point. The 'problem' with object programming is that it can be put together in many, many ways and I would like to have the properties defined ASAP. Eventually I would like to use them even for code compatibility 'on the fly' reconfiguration, like 'MS compatibility' or 'Standard Soap Compatibility' depending on what server you want to talk to from your Java Client.
> >> > > 2.) I am using URL and not Socket connections because SSL Sockets (or proxies) do not seem to work with Sun's SSL1.0.1 release. This is why both HTTPUtils and HTTPSUtils are required. HTTPUtils is used with an http endpoint and HTTPSUtils with an https endpoint, this way we are able to stay 'binary' compatible with the current Apache Soap and need only 2 lines of additional code in the samples (or any existing client code) to make them work over https. An https server is rather complicated to set up and
> >> > > must 'newbie' Soap user will never work in a commercial environment where https is the only way to go. They just want to play and learn Soap with the binary release. They will not need the extra jars from Sun. Most members of the xml-user list have problems even with installing Tomcat and the few jars. IMHO when they become familiar with the http version then by adding the 3 Sun jars they can easily 'graduate' to the Secure Version. There is going to be a very slight increase in overhead but the flexibility
> >> > > of backward compatibility, easy upgrade from the 'training wheels' version to the fully secure 'heavy duty' version is IMHO probably worth it (I am sure you guessed by now that I am and 'automate it' lunatic. One must be to be able to compete with M$$ :-)
> >> > > 3.) re creating the URL just makes me feel comfortable, just in case the incoming URL is miss formed. I have no real strong opinion on this.
> >> > > Any more comments (even violent ones :-) would be much appreciated...... I am trying to organize with my boss' help a code review of this ASAP and I will try to represent your views and let the reviewers see your EMails.
> >> > > Regards - George
> >> > >
> >> > > Nathan Wray wrote:
> >> > >
> >> > > > Hello George;
> >> > > >
> >> > > > I've taken a high level look at the new code. I'll get right to the issues as I see them:
> >> > > >
> >> > > > I'm not sure it's necessary to add 10 in-line calls to the "PropertiesUtils.IniPropertiesUtils()" method to insure it's called. You could pick one key class and add it as static code to the class so it's only called once on initialization rather than several times per SOAP call. I realize it's a do-nothing call after the first one but there is unnecessary per-call overhead to build the call stack and return.
> >> > > >
> >> > > > I think the SOAP code could be done w/o a HTTPSUtils class, and without rebuilding the URL object. I don't see the advantage of the more complicated code and it will certianly hurt performance.
> >> > > >
> >> > > > In short, I'd rather see a few lines in the origional codebase change to add proxy support and to allow an https URL to work correctly, as well as a static SOAP properties loader, and do away with the URL rebuilding and in-line init calls.
> >> > > >
> >> > > > thanks for listening.
> >> > > >
> >> > > > George I Matkovits wrote:
> >> > > >
> >> > > > > Here is the last part. After I finished with the Server Security Additions I will start working with MS-Java compatibility issues. IMHO we should keep Java Soap 'dynamically configurable' with the help of Soap.properties to comply either with the 'open standard' or the de-facto MS standard. MS will keep on changing its implementation (1) to fix its bugs (2) to keep its 'platform' competitor's off balance. I live in a mixed platform environment and we must be able to keep MS and the 'standard' version
> >> > > > > in sink. I am also planning to develop a generic WSTK interface generator for our applications (around 2000-3000 apps) which are currently all web enabled. Please look at the 2 additional 'Secure' samples. Life would have been much easier if the URL class could be sub classed but it is a 'final' class currently.
> >> > > > > Regards - George
> >> > > > > p.s. I start entering this into the Apache CVS but will not commit the changes until your reply comes back. Than you.
> >> >
> >> > --
> >> > Nathan Wray
> >> > nwray@mich.com
> >> > --
> >> > "Can anybody cite a single interesting or
> >> > important idea or argument that's emerged
> >> > from the months of campaigning in the
> >> > current U.S. presidential race?"
> >> > Jon Katz, slashdot.org
> >> > http://slashdot.org/article.pl?sid=00/09/19/035231&mode=nested
> >
> > --
> > Nathan Wray
> > nwray@mich.com
> > --
> > "Can anybody cite a single interesting or
> > important idea or argument that's emerged
> > from the months of campaigning in the
> > current U.S. presidential race?"
> > Jon Katz, slashdot.org
> > http://slashdot.org/article.pl?sid=00/09/19/035231&mode=nested
> >
Re: SSL and Soap + Secure Soap (access validation and credentials
generation/checking)
Posted by George I Matkovits <ma...@uswest.net>.
I am going to try and incorporate any improvements. I am also on my way with the Security enhancements. They both will become part of of the 2.0 source tree when testing is completed. I would be also very happy with any additional suggestions, extensions, improvements, testing (this is OPEN SOURCE! :-). The Security DB will be a flat file like we are using for the Server's Service classes, for efficiency and ease of installation. I gave up on the SQL idea when realized its installation and maintenance problems. Most people on the soap-user forum (or I on Linux) could not manage to install mySQL and JDBC/ODBC drivers without difficulty. I managed to generate my own PKI (128
byte) keys on NT yesterday (for both RSA and DHA) with the Java2/V1.3 APIs. (Now I must also upgrade my Linux System to Sun's V1.3!) This way we will not need Client 'test certificates'. Any production runs will eventually need it. The PKI Generator (Public/Private keys) is just an 'off-line' utility. It takes less then a minute on my 366 MHz Windows2000 System. Verisign is going to kill me. (-:
Regards - George
p.s most of the 'over head' is connection time and now with 'keep alive' operating against Apache or IIS this is minimal for repeated (even SSL) calls. But all and every overhead reduction will count for scale-ability! I will change the the class load to do Properties initialization and creation as you suggested. A great idea, I should have done it that way from the beginning. It would have also reduced test time. I am rebuilding the URL to prevent any possible client screw ups like we had with the 'infamous' Port# bug. I will try and do some kind of 'reuse' also. IMHO it is not rebuilt, even now, when you are not using the AutoSSLEnable but just the SSLEnable request. (Or
something like that, the code is on a different system then my EMAIL.)
Rich Johns wrote:
> Is all this stuff going to become part of the 2.0 source tree? I've lost track of
> what version has what functionality.
>
> thanks.
>
> Nathan Wray wrote:
>
> >
> > George;
> >
> > It's a change to the original codebase that incorporates your Properties file loading and proxy support, and works with HTTP or HTTPS. I'm planning to change it further by removing the in-line string compares and replacing it with class-scope booleans that are set once with static system property checks for performance.
> >
> > I'm concerned that the codebase you're proposing will have a lot of unnecessary overhead per-call. I'm not sure of your motivation in rebuilding the URL object on every call, it seems to be saying that you don't trust the developer to create the URL object they want (and incurring an extra object creation per-call). I intend to have the client side code create a URL object on initialization and use it forever, and having the code re-create it internally for each connection defeats that. Similarly, trying to throw an initialization call into every entry point is wasteful and ultimately you'll miss one. Why not make the call part of the class load with a static block?
> >
> > I might also rename the HTTPUtils class to URLUtils, since it should work with any URL object and it's unnecessarily restrictive to call it HTTP utils.
> >
> > I'll be taking this code into production within the next few months and it needs to be able to handle hundreds of thousands of calls a day peaking at 100+ calls a second, so every extra call, object creation and string concat costs.
> >
> > Respectfully yours
> > Nathan
> >
> >
> > George I Matkovits wrote:
> >
> >> I like changes to changes which simplify the changes and will also help to accomplish the next step: Server Access Control ASAP.
> >> Regards - George
> >>
> >> Nathan Wray wrote:
> >>
> >> > George, what if I send you another version that accomplishes the same results with fewer changes?
> >> >
> >> > George I Matkovits wrote:
> >> >
> >> > > Hello Nathan,
> >> > > Thank you for the comments and just a quick reply. I will do more thinking later off line.
> >> > > 1.) Multiple calls: just in case I missed an entry point. The 'problem' with object programming is that it can be put together in many, many ways and I would like to have the properties defined ASAP. Eventually I would like to use them even for code compatibility 'on the fly' reconfiguration, like 'MS compatibility' or 'Standard Soap Compatibility' depending on what server you want to talk to from your Java Client.
> >> > > 2.) I am using URL and not Socket connections because SSL Sockets (or proxies) do not seem to work with Sun's SSL1.0.1 release. This is why both HTTPUtils and HTTPSUtils are required. HTTPUtils is used with an http endpoint and HTTPSUtils with an https endpoint, this way we are able to stay 'binary' compatible with the current Apache Soap and need only 2 lines of additional code in the samples (or any existing client code) to make them work over https. An https server is rather complicated to set up and
> >> > > must 'newbie' Soap user will never work in a commercial environment where https is the only way to go. They just want to play and learn Soap with the binary release. They will not need the extra jars from Sun. Most members of the xml-user list have problems even with installing Tomcat and the few jars. IMHO when they become familiar with the http version then by adding the 3 Sun jars they can easily 'graduate' to the Secure Version. There is going to be a very slight increase in overhead but the flexibility
> >> > > of backward compatibility, easy upgrade from the 'training wheels' version to the fully secure 'heavy duty' version is IMHO probably worth it (I am sure you guessed by now that I am and 'automate it' lunatic. One must be to be able to compete with M$$ :-)
> >> > > 3.) re creating the URL just makes me feel comfortable, just in case the incoming URL is miss formed. I have no real strong opinion on this.
> >> > > Any more comments (even violent ones :-) would be much appreciated...... I am trying to organize with my boss' help a code review of this ASAP and I will try to represent your views and let the reviewers see your EMails.
> >> > > Regards - George
> >> > >
> >> > > Nathan Wray wrote:
> >> > >
> >> > > > Hello George;
> >> > > >
> >> > > > I've taken a high level look at the new code. I'll get right to the issues as I see them:
> >> > > >
> >> > > > I'm not sure it's necessary to add 10 in-line calls to the "PropertiesUtils.IniPropertiesUtils()" method to insure it's called. You could pick one key class and add it as static code to the class so it's only called once on initialization rather than several times per SOAP call. I realize it's a do-nothing call after the first one but there is unnecessary per-call overhead to build the call stack and return.
> >> > > >
> >> > > > I think the SOAP code could be done w/o a HTTPSUtils class, and without rebuilding the URL object. I don't see the advantage of the more complicated code and it will certianly hurt performance.
> >> > > >
> >> > > > In short, I'd rather see a few lines in the origional codebase change to add proxy support and to allow an https URL to work correctly, as well as a static SOAP properties loader, and do away with the URL rebuilding and in-line init calls.
> >> > > >
> >> > > > thanks for listening.
> >> > > >
> >> > > > George I Matkovits wrote:
> >> > > >
> >> > > > > Here is the last part. After I finished with the Server Security Additions I will start working with MS-Java compatibility issues. IMHO we should keep Java Soap 'dynamically configurable' with the help of Soap.properties to comply either with the 'open standard' or the de-facto MS standard. MS will keep on changing its implementation (1) to fix its bugs (2) to keep its 'platform' competitor's off balance. I live in a mixed platform environment and we must be able to keep MS and the 'standard' version
> >> > > > > in sink. I am also planning to develop a generic WSTK interface generator for our applications (around 2000-3000 apps) which are currently all web enabled. Please look at the 2 additional 'Secure' samples. Life would have been much easier if the URL class could be sub classed but it is a 'final' class currently.
> >> > > > > Regards - George
> >> > > > > p.s. I start entering this into the Apache CVS but will not commit the changes until your reply comes back. Than you.
> >> >
> >> > --
> >> > Nathan Wray
> >> > nwray@mich.com
> >> > --
> >> > "Can anybody cite a single interesting or
> >> > important idea or argument that's emerged
> >> > from the months of campaigning in the
> >> > current U.S. presidential race?"
> >> > Jon Katz, slashdot.org
> >> > http://slashdot.org/article.pl?sid=00/09/19/035231&mode=nested
> >
> > --
> > Nathan Wray
> > nwray@mich.com
> > --
> > "Can anybody cite a single interesting or
> > important idea or argument that's emerged
> > from the months of campaigning in the
> > current U.S. presidential race?"
> > Jon Katz, slashdot.org
> > http://slashdot.org/article.pl?sid=00/09/19/035231&mode=nested
> >
Re: SSL and Soap
Posted by Rich Johns <rj...@vignette.com>.
Is all this stuff going to become part of the 2.0 source tree? I've lost track of
what version has what functionality.
thanks.
Nathan Wray wrote:
>
> George;
>
> It's a change to the original codebase that incorporates your Properties file loading and proxy
> support, and works with HTTP or HTTPS. I'm planning to change it further by removing the in-line
> string compares and replacing it with class-scope booleans that are set once with static system
> property checks for performance.
>
> I'm concerned that the codebase you're proposing will have a lot of unnecessary overhead
> per-call. I'm not sure of your motivation in rebuilding the URL object on every call, it seems to
> be saying that you don't trust the developer to create the URL object they want (and incurring an
> extra object creation per-call). I intend to have the client side code create a URL object on
> initialization and use it forever, and having the code re-create it internally for each connection
> defeats that. Similarly, trying to throw an initialization call into every entry point is
> wasteful and ultimately you'll miss one. Why not make the call part of the class load with a
> static block?
>
> I might also rename the HTTPUtils class to URLUtils, since it should work with any URL object and
> it's unnecessarily restrictive to call it HTTP utils.
>
> I'll be taking this code into production within the next few months and it needs to be able to
> handle hundreds of thousands of calls a day peaking at 100+ calls a second, so every extra call,
> object creation and string concat costs.
>
> Respectfully yours
> Nathan
>
>
> George I Matkovits wrote:
>
>> I like changes to changes which simplify the changes and will also help to accomplish the next
>> step: Server Access Control ASAP.
>> Regards - George
>>
>> Nathan Wray wrote:
>>
>> > George, what if I send you another version that accomplishes the same results with fewer
>> changes?
>> >
>> > George I Matkovits wrote:
>> >
>> > > Hello Nathan,
>> > > Thank you for the comments and just a quick reply. I will do more thinking later off line.
>> > > 1.) Multiple calls: just in case I missed an entry point. The 'problem' with object
>> programming is that it can be put together in many, many ways and I would like to have the
>> properties defined ASAP. Eventually I would like to use them even for code compatibility 'on
>> the fly' reconfiguration, like 'MS compatibility' or 'Standard Soap Compatibility' depending on
>> what server you want to talk to from your Java Client.
>> > > 2.) I am using URL and not Socket connections because SSL Sockets (or proxies) do not seem
>> to work with Sun's SSL1.0.1 release. This is why both HTTPUtils and HTTPSUtils are required.
>> HTTPUtils is used with an http endpoint and HTTPSUtils with an https endpoint, this way we are
>> able to stay 'binary' compatible with the current Apache Soap and need only 2 lines of
>> additional code in the samples (or any existing client code) to make them work over https. An
>> https server is rather complicated to set up and
>> > > must 'newbie' Soap user will never work in a commercial environment where https is the only
>> way to go. They just want to play and learn Soap with the binary release. They will not need the
>> extra jars from Sun. Most members of the xml-user list have problems even with installing Tomcat
>> and the few jars. IMHO when they become familiar with the http version then by adding the 3 Sun
>> jars they can easily 'graduate' to the Secure Version. There is going to be a very slight
>> increase in overhead but the flexibility
>> > > of backward compatibility, easy upgrade from the 'training wheels' version to the fully
>> secure 'heavy duty' version is IMHO probably worth it (I am sure you guessed by now that I am
>> and 'automate it' lunatic. One must be to be able to compete with M$$ :-)
>> > > 3.) re creating the URL just makes me feel comfortable, just in case the incoming URL is
>> miss formed. I have no real strong opinion on this.
>> > > Any more comments (even violent ones :-) would be much appreciated...... I am trying to
>> organize with my boss' help a code review of this ASAP and I will try to represent your views
>> and let the reviewers see your EMails.
>> > > Regards - George
>> > >
>> > > Nathan Wray wrote:
>> > >
>> > > > Hello George;
>> > > >
>> > > > I've taken a high level look at the new code. I'll get right to the issues as I see them:
>>
>> > > >
>> > > > I'm not sure it's necessary to add 10 in-line calls to the
>> "PropertiesUtils.IniPropertiesUtils()" method to insure it's called. You could pick one key
>> class and add it as static code to the class so it's only called once on initialization rather
>> than several times per SOAP call. I realize it's a do-nothing call after the first one but
>> there is unnecessary per-call overhead to build the call stack and return.
>> > > >
>> > > > I think the SOAP code could be done w/o a HTTPSUtils class, and without rebuilding the URL
>> object. I don't see the advantage of the more complicated code and it will certianly hurt
>> performance.
>> > > >
>> > > > In short, I'd rather see a few lines in the origional codebase change to add proxy support
>> and to allow an https URL to work correctly, as well as a static SOAP properties loader, and do
>> away with the URL rebuilding and in-line init calls.
>> > > >
>> > > > thanks for listening.
>> > > >
>> > > > George I Matkovits wrote:
>> > > >
>> > > > > Here is the last part. After I finished with the Server Security Additions I will start
>> working with MS-Java compatibility issues. IMHO we should keep Java Soap 'dynamically
>> configurable' with the help of Soap.properties to comply either with the 'open standard' or
>> the de-facto MS standard. MS will keep on changing its implementation (1) to fix its bugs (2) to
>> keep its 'platform' competitor's off balance. I live in a mixed platform environment and we must
>> be able to keep MS and the 'standard' version
>> > > > > in sink. I am also planning to develop a generic WSTK interface generator for our
>> applications (around 2000-3000 apps) which are currently all web enabled. Please look at the 2
>> additional 'Secure' samples. Life would have been much easier if the URL class could be sub
>> classed but it is a 'final' class currently.
>> > > > > Regards - George
>> > > > > p.s. I start entering this into the Apache CVS but will not commit the changes until
>> your reply comes back. Than you.
>> >
>> > --
>> > Nathan Wray
>> > nwray@mich.com
>> > --
>> > "Can anybody cite a single interesting or
>> > important idea or argument that's emerged
>> > from the months of campaigning in the
>> > current U.S. presidential race?"
>> > Jon Katz, slashdot.org
>> > http://slashdot.org/article.pl?sid=00/09/19/035231&mode=nested
>
> --
> Nathan Wray
> nwray@mich.com
> --
> "Can anybody cite a single interesting or
> important idea or argument that's emerged
> from the months of campaigning in the
> current U.S. presidential race?"
> Jon Katz, slashdot.org
> http://slashdot.org/article.pl?sid=00/09/19/035231&mode=nested
>
Re: SSL and Soap
Posted by Rich Johns <rj...@vignette.com>.
Is all this stuff going to become part of the 2.0 source tree? I've lost track of
what version has what functionality.
thanks.
Nathan Wray wrote:
>
> George;
>
> It's a change to the original codebase that incorporates your Properties file loading and proxy
> support, and works with HTTP or HTTPS. I'm planning to change it further by removing the in-line
> string compares and replacing it with class-scope booleans that are set once with static system
> property checks for performance.
>
> I'm concerned that the codebase you're proposing will have a lot of unnecessary overhead
> per-call. I'm not sure of your motivation in rebuilding the URL object on every call, it seems to
> be saying that you don't trust the developer to create the URL object they want (and incurring an
> extra object creation per-call). I intend to have the client side code create a URL object on
> initialization and use it forever, and having the code re-create it internally for each connection
> defeats that. Similarly, trying to throw an initialization call into every entry point is
> wasteful and ultimately you'll miss one. Why not make the call part of the class load with a
> static block?
>
> I might also rename the HTTPUtils class to URLUtils, since it should work with any URL object and
> it's unnecessarily restrictive to call it HTTP utils.
>
> I'll be taking this code into production within the next few months and it needs to be able to
> handle hundreds of thousands of calls a day peaking at 100+ calls a second, so every extra call,
> object creation and string concat costs.
>
> Respectfully yours
> Nathan
>
>
> George I Matkovits wrote:
>
>> I like changes to changes which simplify the changes and will also help to accomplish the next
>> step: Server Access Control ASAP.
>> Regards - George
>>
>> Nathan Wray wrote:
>>
>> > George, what if I send you another version that accomplishes the same results with fewer
>> changes?
>> >
>> > George I Matkovits wrote:
>> >
>> > > Hello Nathan,
>> > > Thank you for the comments and just a quick reply. I will do more thinking later off line.
>> > > 1.) Multiple calls: just in case I missed an entry point. The 'problem' with object
>> programming is that it can be put together in many, many ways and I would like to have the
>> properties defined ASAP. Eventually I would like to use them even for code compatibility 'on
>> the fly' reconfiguration, like 'MS compatibility' or 'Standard Soap Compatibility' depending on
>> what server you want to talk to from your Java Client.
>> > > 2.) I am using URL and not Socket connections because SSL Sockets (or proxies) do not seem
>> to work with Sun's SSL1.0.1 release. This is why both HTTPUtils and HTTPSUtils are required.
>> HTTPUtils is used with an http endpoint and HTTPSUtils with an https endpoint, this way we are
>> able to stay 'binary' compatible with the current Apache Soap and need only 2 lines of
>> additional code in the samples (or any existing client code) to make them work over https. An
>> https server is rather complicated to set up and
>> > > must 'newbie' Soap user will never work in a commercial environment where https is the only
>> way to go. They just want to play and learn Soap with the binary release. They will not need the
>> extra jars from Sun. Most members of the xml-user list have problems even with installing Tomcat
>> and the few jars. IMHO when they become familiar with the http version then by adding the 3 Sun
>> jars they can easily 'graduate' to the Secure Version. There is going to be a very slight
>> increase in overhead but the flexibility
>> > > of backward compatibility, easy upgrade from the 'training wheels' version to the fully
>> secure 'heavy duty' version is IMHO probably worth it (I am sure you guessed by now that I am
>> and 'automate it' lunatic. One must be to be able to compete with M$$ :-)
>> > > 3.) re creating the URL just makes me feel comfortable, just in case the incoming URL is
>> miss formed. I have no real strong opinion on this.
>> > > Any more comments (even violent ones :-) would be much appreciated...... I am trying to
>> organize with my boss' help a code review of this ASAP and I will try to represent your views
>> and let the reviewers see your EMails.
>> > > Regards - George
>> > >
>> > > Nathan Wray wrote:
>> > >
>> > > > Hello George;
>> > > >
>> > > > I've taken a high level look at the new code. I'll get right to the issues as I see them:
>>
>> > > >
>> > > > I'm not sure it's necessary to add 10 in-line calls to the
>> "PropertiesUtils.IniPropertiesUtils()" method to insure it's called. You could pick one key
>> class and add it as static code to the class so it's only called once on initialization rather
>> than several times per SOAP call. I realize it's a do-nothing call after the first one but
>> there is unnecessary per-call overhead to build the call stack and return.
>> > > >
>> > > > I think the SOAP code could be done w/o a HTTPSUtils class, and without rebuilding the URL
>> object. I don't see the advantage of the more complicated code and it will certianly hurt
>> performance.
>> > > >
>> > > > In short, I'd rather see a few lines in the origional codebase change to add proxy support
>> and to allow an https URL to work correctly, as well as a static SOAP properties loader, and do
>> away with the URL rebuilding and in-line init calls.
>> > > >
>> > > > thanks for listening.
>> > > >
>> > > > George I Matkovits wrote:
>> > > >
>> > > > > Here is the last part. After I finished with the Server Security Additions I will start
>> working with MS-Java compatibility issues. IMHO we should keep Java Soap 'dynamically
>> configurable' with the help of Soap.properties to comply either with the 'open standard' or
>> the de-facto MS standard. MS will keep on changing its implementation (1) to fix its bugs (2) to
>> keep its 'platform' competitor's off balance. I live in a mixed platform environment and we must
>> be able to keep MS and the 'standard' version
>> > > > > in sink. I am also planning to develop a generic WSTK interface generator for our
>> applications (around 2000-3000 apps) which are currently all web enabled. Please look at the 2
>> additional 'Secure' samples. Life would have been much easier if the URL class could be sub
>> classed but it is a 'final' class currently.
>> > > > > Regards - George
>> > > > > p.s. I start entering this into the Apache CVS but will not commit the changes until
>> your reply comes back. Than you.
>> >
>> > --
>> > Nathan Wray
>> > nwray@mich.com
>> > --
>> > "Can anybody cite a single interesting or
>> > important idea or argument that's emerged
>> > from the months of campaigning in the
>> > current U.S. presidential race?"
>> > Jon Katz, slashdot.org
>> > http://slashdot.org/article.pl?sid=00/09/19/035231&mode=nested
>
> --
> Nathan Wray
> nwray@mich.com
> --
> "Can anybody cite a single interesting or
> important idea or argument that's emerged
> from the months of campaigning in the
> current U.S. presidential race?"
> Jon Katz, slashdot.org
> http://slashdot.org/article.pl?sid=00/09/19/035231&mode=nested
>