Crypto and initialisation

[Graham Leggett <mi...@sharp.fm>]

Hi all,

Having just finished the next iteration of abstracted crypto support for 
APR (currently on apr-util-trunk), a problem has cropped up with the 
assumptions made by mod_ssl and mod_nss.

To date, both mod_ssl and mod_nss have made the assumption that they 
will be the only crypto modules loaded into the server, and so have 
"owned" the task of crypto initialisation.

OpenSSL seems to be tolerant of being initialised twice, and so it has 
been possible for mod_ssl, mod_session_crypto and the external 
mod_auth_openid to coexist within the same server and this has worked, 
but by accident.

The same cannot be said however for NSS - NSS requires that a crypto 
database be specified on initialisation, and if two modules tried to 
initialise NSS independently of each other, much confusion and 
brokenness will result.

What I propose to do to fix this for v2.4 and beyond is write a simple 
module mod_crypto whose job it is to initialise the user's chosen 
crypto(s) at most once, and serve as a parent module to mod_ssl and any 
other crypto module that wants to play.

mod_ssl, mod_nss, mod_session_crypto, mod_auth_openid and friends will 
then have a single mechanism to determine whether crypto has been 
configured successfully, and which crypto libraries are involved, and 
none of these modules need stomp on any of the others.

Before embarking on this task, I would like to check whether people 
believe this to be the optimal solution to the problem, or whether a 
better solution exists.

Thoughts?

Regards,
Graham
--

Re: Crypto and initialisation

[Graham Leggett <mi...@sharp.fm>]

William A. Rowe, Jr. wrote:

> They are; and shouldn't be used.  pszProvider and dwProvType args of
> CryptAcquireContext are the function you were looking for.  This returns
> a stateful, freethreaded handle for that module's access.
> 
> Just because their remains bogus app-default functions doesn't mean it's
> wise to use them.  And this function doesn't seem relevant to the "init"
> function you called out in the first place, was it?

You tell me.

All I have available to me on the CAPI side is the MS documentation and 
a number of examples of how people have abstracted the CAPI API.

The MS docs don't tell me which functions are "bogus", or which 
functions are "wise to use". And another end user of the abstraction 
might disagree as to what is bogus and what is not.

This is why I started this thread: to pick up flaws in my understanding 
so that they may be corrected, so I don't go off and waste my time 
writing a whole lot of code which will be rejected out of hand, and to 
come up with a workable compromise to solve this problem.

By picking through the NSS code, I have learned more about the behaviour 
of their init process. The first entity to initialise NSS wins. All 
further attempts to initialise silently succeed. This means NSS can be 
safely double or triple initialised, but it also means that if different 
parameters are used during the second or third attempt to initialise, 
those parameters will be silently ignored.

This means an end user problem still remains. Right now, if three NSS 
based modules came along, with three separate sets of directives to 
initialise NSS, the user might conclude that it was possible to 
initialise each module against three different databases. As it turns 
out, one of the three will win, and which one will depend on their 
module load order, and that is not intuitive to an end user.

How do you suggest I solve this problem?

Regards,
Graham
--

Re: Crypto and initialisation

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Graham Leggett wrote:
> William A. Rowe, Jr. wrote:
> 
>> Nonsense.  In the MS world, double initialization and thread safety are
>> entirely mandated by design.
> 
> So, if I am reading you correctly, if module A called 
> CryptGetDefaultProvider, then set parameters using CryptSetProvider or 
> CryptSetProvParam, and then module B came along and did the same thing, 
> these two modules would have completely independent handles to crypto 
> that would not clash with each other? The docs I have read so far imply 
> that settings are application wide. Can you confirm?

They are; and shouldn't be used.  pszProvider and dwProvType args of
CryptAcquireContext are the function you were looking for.  This returns
a stateful, freethreaded handle for that module's access.

Just because their remains bogus app-default functions doesn't mean it's
wise to use them.  And this function doesn't seem relevant to the "init"
function you called out in the first place, was it?

Re: Crypto and initialisation

[Graham Leggett <mi...@sharp.fm>]

William A. Rowe, Jr. wrote:

> Nonsense.  In the MS world, double initialization and thread safety are
> entirely mandated by design.

So, if I am reading you correctly, if module A called 
CryptGetDefaultProvider, then set parameters using CryptSetProvider or 
CryptSetProvParam, and then module B came along and did the same thing, 
these two modules would have completely independent handles to crypto 
that would not clash with each other? The docs I have read so far imply 
that settings are application wide. Can you confirm?

> Once upon a time, there was a need for a program to accept SSL input and
> do something other with it.  A single use of SSL and nothing further.
 >
> Today, the program which accepts SSL input then needs to ask questions of
> several other programs in an unconnected manner, also using SSL.  Take 
> auth,
> for example.  Even inquiring about the current processes credentials
> (think nss/pam/ldap) is going to trigger the necessity of an SSL provider.
> 
> So any SSL provider incapable of double initialization today is irrelevant
> from its inception.  That includes nss; perhaps this is a significant 
> reason
> it's never been more widely adopted?  My experiences using nss-based ldap
> have not been particularly fun in terms of interop.

Doesn't look that way to me:

http://fedoraproject.org/wiki/FedoraCryptoConsolidation

Let me chase up the NSS people and see what their comments are.

Regards,
Graham
--

Re: Crypto and initialisation

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Graham Leggett wrote:
> Paul Querna wrote:
> 
>> Then the API is broken.
>>
>> OpenSSL and GnuTLS both allow 'double' initialization, as long as they 
>> are also deinitiilzed the same number of times, just like APR does too.
>>
>>> What I propose to do to fix this for v2.4 and beyond is write a 
>>> simple module mod_crypto whose job it is to initialise the user's 
>>> chosen crypto(s) at most once, and serve as a parent module to 
>>> mod_ssl and any other crypto module that wants to play.
>>
>> Make the API authors fix their APIs, don't add another module.
> 
> I suspect we are 12 years too late on this particular issue. While the 
> httpd project certainly is well known, I would be pressed to think that 
> Microsoft or NSS will change their long established APIs on our account.

Nonsense.  In the MS world, double initialization and thread safety are
entirely mandated by design.

Once upon a time, there was a need for a program to accept SSL input and
do something other with it.  A single use of SSL and nothing further.

Today, the program which accepts SSL input then needs to ask questions of
several other programs in an unconnected manner, also using SSL.  Take auth,
for example.  Even inquiring about the current processes credentials
(think nss/pam/ldap) is going to trigger the necessity of an SSL provider.

So any SSL provider incapable of double initialization today is irrelevant
from its inception.  That includes nss; perhaps this is a significant reason
it's never been more widely adopted?  My experiences using nss-based ldap
have not been particularly fun in terms of interop.

Bill

Re: Crypto and initialisation

[Graham Leggett <mi...@sharp.fm>]

Paul Querna wrote:

> Then the API is broken.
> 
> OpenSSL and GnuTLS both allow 'double' initialization, as long as they 
> are also deinitiilzed the same number of times, just like APR does too.
> 
>> What I propose to do to fix this for v2.4 and beyond is write a simple 
>> module mod_crypto whose job it is to initialise the user's chosen 
>> crypto(s) at most once, and serve as a parent module to mod_ssl and 
>> any other crypto module that wants to play.
> 
> Make the API authors fix their APIs, don't add another module.

I suspect we are 12 years too late on this particular issue. While the 
httpd project certainly is well known, I would be pressed to think that 
Microsoft or NSS will change their long established APIs on our account.

If OpenSSL does support double initialisation (this capability isn't 
mentioned in the OpenSSL API), then the problem is reduced to modules 
using NSS or CAPI only, which means that the scope of the initialisation 
module would reduce to modules using the crypto abstraction layer only, 
and mod_ssl can be left alone.

This does sound like a cleaner approach.

Regards,
Graham
--

Re: Crypto and initialisation

[Paul Querna <ch...@force-elite.com>]

Graham Leggett wrote:
> Hi all,
> 
> Having just finished the next iteration of abstracted crypto support for 
> APR (currently on apr-util-trunk), a problem has cropped up with the 
> assumptions made by mod_ssl and mod_nss.
> 
> To date, both mod_ssl and mod_nss have made the assumption that they 
> will be the only crypto modules loaded into the server, and so have 
> "owned" the task of crypto initialisation.
> 
> OpenSSL seems to be tolerant of being initialised twice, and so it has 
> been possible for mod_ssl, mod_session_crypto and the external 
> mod_auth_openid to coexist within the same server and this has worked, 
> but by accident.
> 
> The same cannot be said however for NSS - NSS requires that a crypto 
> database be specified on initialisation, and if two modules tried to 
> initialise NSS independently of each other, much confusion and 
> brokenness will result.

Then the API is broken.

OpenSSL and GnuTLS both allow 'double' initialization, as long as they 
are also deinitiilzed the same number of times, just like APR does too.

> What I propose to do to fix this for v2.4 and beyond is write a simple 
> module mod_crypto whose job it is to initialise the user's chosen 
> crypto(s) at most once, and serve as a parent module to mod_ssl and any 
> other crypto module that wants to play.

Make the API authors fix their APIs, don't add another module.

-Paul