You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Joe Orton <jo...@redhat.com> on 2008/02/25 23:12:43 UTC
RFC: extracting the mod_ssl session cache interface
The session cache interface in mod_ssl on the trunk is now mostly SSL-
and mod_ssl-agnostic with respect to the data storage and configuration.
There is still some tight coupling between the session cache and the
ssl_mutex interface, but that's next on my hit list.
Is there any interest in seeing this extracted from mod_ssl and made
available for general use? It could probably e.g. be used by
mod_auth_digest for the MD5-sess code, and I can think of some
third-party modules which could probably use it too (mod_gnutls).
My vague plan would be to finish de-SSL-ifying the code, then moving it
to modules/cache and calling it mod_sesscache or mod_socache ("small
object") or something along those lines.
Thoughts?
joe
Re: RFC: extracting the mod_ssl session cache interface
Posted by Issac Goldstand <ma...@beamartyr.net>.
Justin Erenkrantz wrote:
> On Mon, Feb 25, 2008 at 2:12 PM, Joe Orton <jo...@redhat.com> wrote:
>> The session cache interface in mod_ssl on the trunk is now mostly SSL-
>> and mod_ssl-agnostic with respect to the data storage and configuration.
>> There is still some tight coupling between the session cache and the
>> ssl_mutex interface, but that's next on my hit list.
>>
>> Is there any interest in seeing this extracted from mod_ssl and made
>> available for general use? It could probably e.g. be used by
>> mod_auth_digest for the MD5-sess code, and I can think of some
>> third-party modules which could probably use it too (mod_gnutls).
>>
>> My vague plan would be to finish de-SSL-ifying the code, then moving it
>> to modules/cache and calling it mod_sesscache or mod_socache ("small
>> object") or something along those lines.
>
> "mod_small_cache"?
>
-0
It sounds like that *Should* be mod_cache_small (which will muck up the
cache namespace). I'd rather see one longish name like Joe suggested
than leave "cache" on its own
+1 to the idea in general. If the cache isn't memory-only based, would
be nice to see this done in apr-dbd for easy porting of backend storage
systems...
Re: RFC: extracting the mod_ssl session cache interface
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
On Mon, Feb 25, 2008 at 2:12 PM, Joe Orton <jo...@redhat.com> wrote:
> The session cache interface in mod_ssl on the trunk is now mostly SSL-
> and mod_ssl-agnostic with respect to the data storage and configuration.
> There is still some tight coupling between the session cache and the
> ssl_mutex interface, but that's next on my hit list.
>
> Is there any interest in seeing this extracted from mod_ssl and made
> available for general use? It could probably e.g. be used by
> mod_auth_digest for the MD5-sess code, and I can think of some
> third-party modules which could probably use it too (mod_gnutls).
>
> My vague plan would be to finish de-SSL-ifying the code, then moving it
> to modules/cache and calling it mod_sesscache or mod_socache ("small
> object") or something along those lines.
"mod_small_cache"?
> Thoughts?
In general, it sounds like a cool plan. -- justin
Re: RFC: extracting the mod_ssl session cache interface
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Feb 25, 2008, at 5:12 PM, Joe Orton wrote:
> The session cache interface in mod_ssl on the trunk is now mostly SSL-
> and mod_ssl-agnostic with respect to the data storage and
> configuration.
> There is still some tight coupling between the session cache and the
> ssl_mutex interface, but that's next on my hit list.
>
> Is there any interest in seeing this extracted from mod_ssl and made
> available for general use? It could probably e.g. be used by
> mod_auth_digest for the MD5-sess code, and I can think of some
> third-party modules which could probably use it too (mod_gnutls).
>
> My vague plan would be to finish de-SSL-ifying the code, then moving
> it
> to modules/cache and calling it mod_sesscache or mod_socache ("small
> object") or something along those lines.
>
+1
How about mod_gicache (for generic internal cache)? Otherwise
mod_socache
is good. I'm expecting that it's a generic internal cache and
naming it sesscache may confuse the issue.
Re: RFC: extracting the mod_ssl session cache interface
Posted by Chris Darroch <ch...@pearsoncmg.com>.
Joe Orton wrote:
> Is there any interest in seeing this extracted from mod_ssl and made
> available for general use? It could probably e.g. be used by
> mod_auth_digest for the MD5-sess code, and I can think of some
> third-party modules which could probably use it too (mod_gnutls).
>
> My vague plan would be to finish de-SSL-ifying the code, then moving it
> to modules/cache and calling it mod_sesscache or mod_socache ("small
> object") or something along those lines.
Definitely +1 from me. If you don't mind, please have a glance
over my thoughts from last year on similar kinds of issues:
http://svn.apache.org/viewvc/httpd/sandbox/amsterdam/architecture/scoreboard.txt?view=markup
At the bottom of all that I'd flagged mod_ssl and mod_auth_digest
as possible users of a generic cache mechanism, as well the scoreboard
and mod_proxy.
Roughly what I'd envisioned was several "levels" of providers,
each of which must provide a small set of hash (atomic) and table
(possibly non-atomic) access functions.
The intention of the table access functions is, primarily, that
these would fulfill the needs of the scoreboard: an integer-based table
of fixed-length records, which if stored in shared memory would be
updated non-atomically to ensure relatively quick access. Essentially,
what the scoreboard does now, but re-factored.
Leaving the scoreboard aside, most other potential users might
prefer the hash access functions; slower, but guaranteed to be
atomic. I hadn't thought about the format of keys and values, but
since Paul Querna asked about it, I'd be inclined to suggest
perhaps strings or apr_datam_t for keys and apr_datum_t for values.
String keys allow for prefixes, as others suggested.
I tried to describe some possible providers, with the "simpler"
ones being just wrappers around DBD or DBM or plain APR tables/hashes
(that last being only appropriate for single-process servers).
The next step up -- shy of a whizz-bang distributed system
supporting multiple httpd servers -- would likely be the key provider
to implement first, one that used shared memory. In some ways this is
the hardest, because of the issues around sizing the memory appropriately
and dealing with allocations within it.
I proposed a few notions about how such a provider and its
users might interact, with the provider indicating it could not
support resizing, users (i.e., modules) indicating their requirements,
and the server coordinating with the provider at startup and restart
time to request one large block and then handling out hashes and
tables to the various users.
Resizing the allocated shared memory when restarting (for example,
because a module has been added to the server configs, or some directives
were changed) is pretty unpleasant. The scoreboard currently just dodges
the problem by sizing only at startup, from compile-time maximums. At
the bottom of my notes there are some ramblings about how one might
reference count the allocations from previous server generations and
deallocate them when all child processes of that generated had exited;
I can see this might prove unworkable in real life, though.
For many non-shared-memory based providers like DBM, DBD, plain APR,
or some external cache (memcache, etc.), resizing should be trivial,
of course.
I happen to have some breathing room coming up in between other
things and would be keen to toss around some ideas and/or code.
I'm open to suggestions on how best to proceed. Maybe some
mocked-up header files to discuss function prototypes? Some stub
code in amsterdam or elsewhere? Further discussion? Please feel
free to add/amend/flame my notes in that scoreboard.txt file!
Chris.
--
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: RFC: extracting the mod_ssl session cache interface
Posted by Matthieu Estrade <me...@apache.org>.
Joe Orton wrote:
> The session cache interface in mod_ssl on the trunk is now mostly SSL-
> and mod_ssl-agnostic with respect to the data storage and configuration.
> There is still some tight coupling between the session cache and the
> ssl_mutex interface, but that's next on my hit list.
>
> Is there any interest in seeing this extracted from mod_ssl and made
> available for general use? It could probably e.g. be used by
> mod_auth_digest for the MD5-sess code, and I can think of some
> third-party modules which could probably use it too (mod_gnutls).
>
> My vague plan would be to finish de-SSL-ifying the code, then moving it
> to modules/cache and calling it mod_sesscache or mod_socache ("small
> object") or something along those lines.
>
> Thoughts?
>
> joe
>
I didn't investigate this a lot but using ssl session in mod_proxy when
doing reverse proxy to connect to backend could be very useful.
Using the client -> apache ssl session id could be interesting to keep
synchronisation between the two side.
Matthieu
Re: mod_ssl & CRL verification
Posted by Dr Stephen Henson <sh...@oss-institute.org>.
Joe Orton wrote:
> On Tue, Feb 26, 2008 at 04:51:40PM +0000, Dr Stephen Henson wrote:
>> Well the current CRL strategy has a few problems. It ignores critical
>> extensions but that's a separate issue...
>
> I was looking at this recently; is it still true that mod_ssl has to do
> so much of the CRL revocation checks for client certs itself (i.e. all
> of ssl_callback_SSLVerify_CRL) - it looks like X509_verify_cert() can do
> revocation checks itself if suitably configured, though maybe this is a
> recent addition?
>
Some enhanced CRL support in X509_verify_cert() has been in OpenSSL for
some time (over a year).
You just need to set the relevant flags and OpenSSL will handle things.
OpenSSL 0.9.7 checks for critical CRL extensions and rejects a CRL if it
finds any.
0.9.8 can also use key identifiers to look up CRLs.
0.9.9 also includes support for extensions like IDP for CRL
partitioning. It also allows multiple CRLs with the same scope to appear
in a store and uses the first valid one (likely to change that to most
recent). There is also a form of dynamic CRL loading. The functionality
will be extended in future.
There is a difference in the directory handling. OpenSSL doesn't make
any distinction between certificate and CRL directories: a CRL can
appear in a certificate directory and vice-versa.
Steve.
--
Dr Stephen N. Henson. Senior Technical/Cryptography Advisor,
Open Source Software Institute: www.oss-institute.org
OpenSSL Core team: www.openssl.org
Re: mod_ssl & CRL verification
Posted by Eric Covener <co...@gmail.com>.
On Tue, Feb 26, 2008 at 1:57 PM, Joe Orton <jo...@redhat.com> wrote:
> Right, that is exactly my view. I think that any attempt to make
> mod_ssl treat CRLs as anything other than static files loaded once at
> startup will end up trying to reinvent OCSP badly.
>
> If a free OCSP responder existed which actually did this maybe those
> "make CRL handling better" bug reports would go away :)
FWIW I have experimented with this recently and found ocspd from
openca.org was able to frontend a CRL-as-static-file
satisfactorily (albeit for a different security library and SSL
application). It seems to be BSD-like and gratis.
Unfortunately I stopped short of trying to frontend a CRL-over-LDAP,
but it does purport to do this as part of its core functionality.
https://www.openca.org/projects/ocspd/downloads.shtml
--
Eric Covener
covener@gmail.com
mod_ssl & CRL verification
Posted by Joe Orton <jo...@redhat.com>.
On Tue, Feb 26, 2008 at 04:51:40PM +0000, Dr Stephen Henson wrote:
> Well the current CRL strategy has a few problems. It ignores critical
> extensions but that's a separate issue...
I was looking at this recently; is it still true that mod_ssl has to do
so much of the CRL revocation checks for client certs itself (i.e. all
of ssl_callback_SSLVerify_CRL) - it looks like X509_verify_cert() can do
revocation checks itself if suitably configured, though maybe this is a
recent addition?
> Many CRLs have short lifetimes and need to be updated fairly often which
> causes problems when the server needs to be restarted each time.
...
> Well that's one strategy... another would be to use OCSP exclusively and
> have a local OCSP responder driven by CRLs.
Right, that is exactly my view. I think that any attempt to make
mod_ssl treat CRLs as anything other than static files loaded once at
startup will end up trying to reinvent OCSP badly.
If a free OCSP responder existed which actually did this maybe those
"make CRL handling better" bug reports would go away :)
joe
Re: RFC: extracting the mod_ssl session cache interface
Posted by Dr Stephen Henson <sh...@oss-institute.org>.
Joe Orton wrote:
> On Mon, Feb 25, 2008 at 10:54:58PM +0000, Dr Stephen Henson wrote:
>
>> If it could hold (potentially) larger objects or large numbers of small
>> objects then it could help make the CRL code more usable.
>
> I'm not sure exactly what you're referring to there (caching CRL lookup
> results?), but it depends on what exactly you mean by "large" and
> "small", in any case. shmcb might need to be tuned differently to be
> useful for caching small numbers of large objects; Google says memcache
> will handle objects up to 1MB by default; so, quite "large".
>
Well the current CRL strategy has a few problems. It ignores critical
extensions but that's a separate issue...
Many CRLs have short lifetimes and need to be updated fairly often which
causes problems when the server needs to be restarted each time.
Some CRLs can be quite large, running into several MB.
In a multi process server it isn't very efficient loading a large CRL
each time it is needed.
The encoded CRL could be shared between all processes but that still
results in an overhead in that the CRL would need to be parsed and
verified repeatedly.
A better strategy is to load and verify the CRL *once* and then index
the revoked certificate entries.
Then subsequent processes just need to check to see if a revoked entry
exists for the client certificate in question.
Loading in a new CRL would need to lock the whole thing while it was
updated of course.
Well that's one strategy... another would be to use OCSP exclusively and
have a local OCSP responder driven by CRLs.
Steve.
--
Dr Stephen N. Henson. Senior Technical/Cryptography Advisor,
Open Source Software Institute: www.oss-institute.org
OpenSSL Core team: www.openssl.org
Re: RFC: extracting the mod_ssl session cache interface
Posted by Joe Orton <jo...@redhat.com>.
On Mon, Feb 25, 2008 at 10:54:58PM +0000, Dr Stephen Henson wrote:
> Well I can think of a several applications for de-SSL-ifying or
> specifically de-SSL_SESSION-ifying (i.e. being able to store things other
> than SSL_SESSION) the code straight off.
>
> Both SSL related.
>
> The OCSP stapling patch hacked the cached OCSP response data into an
> SSL_SESSION structure to minimised the changes. A general purpose object
> cache would make it much cleaner.
Yeah, this was part of my motivation for doing this work.
> If it could hold (potentially) larger objects or large numbers of small
> objects then it could help make the CRL code more usable.
I'm not sure exactly what you're referring to there (caching CRL lookup
results?), but it depends on what exactly you mean by "large" and
"small", in any case. shmcb might need to be tuned differently to be
useful for caching small numbers of large objects; Google says memcache
will handle objects up to 1MB by default; so, quite "large".
shmcb could probably take hints from the cache consumer on average
object size & count to partition the cache appropriately.
joe
Re: RFC: extracting the mod_ssl session cache interface
Posted by Dr Stephen Henson <sh...@oss-institute.org>.
Joe Orton wrote:
> The session cache interface in mod_ssl on the trunk is now mostly SSL-
> and mod_ssl-agnostic with respect to the data storage and configuration.
> There is still some tight coupling between the session cache and the
> ssl_mutex interface, but that's next on my hit list.
>
> Is there any interest in seeing this extracted from mod_ssl and made
> available for general use? It could probably e.g. be used by
> mod_auth_digest for the MD5-sess code, and I can think of some
> third-party modules which could probably use it too (mod_gnutls).
>
> My vague plan would be to finish de-SSL-ifying the code, then moving it
> to modules/cache and calling it mod_sesscache or mod_socache ("small
> object") or something along those lines.
>
Well I can think of a several applications for de-SSL-ifying or
specifically de-SSL_SESSION-ifying (i.e. being able to store things
other than SSL_SESSION) the code straight off.
Both SSL related.
The OCSP stapling patch hacked the cached OCSP response data into an
SSL_SESSION structure to minimised the changes. A general purpose object
cache would make it much cleaner.
If it could hold (potentially) larger objects or large numbers of small
objects then it could help make the CRL code more usable.
Steve.
--
Dr Stephen N. Henson. Senior Technical/Cryptography Advisor,
Open Source Software Institute: www.oss-institute.org
OpenSSL Core team: www.openssl.org
Re: RFC: extracting the mod_ssl session cache interface
Posted by Ruediger Pluem <rp...@apache.org>.
On 02/25/2008 11:12 PM, Joe Orton wrote:
> The session cache interface in mod_ssl on the trunk is now mostly SSL-
> and mod_ssl-agnostic with respect to the data storage and configuration.
> There is still some tight coupling between the session cache and the
> ssl_mutex interface, but that's next on my hit list.
>
> Is there any interest in seeing this extracted from mod_ssl and made
> available for general use? It could probably e.g. be used by
> mod_auth_digest for the MD5-sess code, and I can think of some
> third-party modules which could probably use it too (mod_gnutls).
>
> My vague plan would be to finish de-SSL-ifying the code, then moving it
> to modules/cache and calling it mod_sesscache or mod_socache ("small
> object") or something along those lines.
Sounds reasonable and good.
Regards
Rüdiger
Re: RFC: extracting the mod_ssl session cache interface
Posted by Joe Orton <jo...@redhat.com>.
On Mon, Feb 25, 2008 at 05:26:08PM -0800, Paul Querna wrote:
> how did you want to structure keys?
Interesting question. I envisage this working by having consumers
configure and set up independent cache instances; so if you had
mod_auth_digest and mod_ssl both using a shmcb provider, you'd get two
shm segments, for example.
For dbm/shmcb you just get a unique key/value pair cache and don't
really have to worry about the key namespace.
For memcache, I presume that it's normal to share a particular server
cluster for caching independent sets of data? The interface could
require that the consumer provides a namespace string which gets
automatically prefixed to the key names in the memcache provider.
Makes sense?
joe
Re: RFC: extracting the mod_ssl session cache interface
Posted by Paul Querna <ch...@force-elite.com>.
Joe Orton wrote:
> The session cache interface in mod_ssl on the trunk is now mostly SSL-
> and mod_ssl-agnostic with respect to the data storage and configuration.
> There is still some tight coupling between the session cache and the
> ssl_mutex interface, but that's next on my hit list.
>
> Is there any interest in seeing this extracted from mod_ssl and made
> available for general use? It could probably e.g. be used by
> mod_auth_digest for the MD5-sess code, and I can think of some
> third-party modules which could probably use it too (mod_gnutls).
>
> My vague plan would be to finish de-SSL-ifying the code, then moving it
> to modules/cache and calling it mod_sesscache or mod_socache ("small
> object") or something along those lines.
>
> Thoughts?
+1 in general.
how did you want to structure keys?