You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Stefan Eissing <st...@greenbytes.de> on 2016/07/11 10:25:19 UTC
concurrent ssl context access
In https://bz.apache.org/bugzilla/show_bug.cgi?id=59840 the issue crept up that StdEnvVars makes concurrent access to the SSL context when HTTP/2 is in place.
The question is: how do we want to fix this?
The general theme is that a module needs to perform some action on a connection, sees that it is a slave connection and accesses the master instead. Such a module needs to be aware that access to master may happen in parallel with other request.
Should we let each such module care for it on its own or do we want to provide a way to access the master connection in a mutex protected way?
-Stefan
Re: concurrent ssl context access
Posted by Stefan Eissing <st...@greenbytes.de>.
> Am 12.07.2016 um 15:28 schrieb William A Rowe Jr <wr...@rowe-clan.net>:
>
> On Tue, Jul 12, 2016 at 3:37 AM, Stefan Eissing <st...@greenbytes.de> wrote:
>
> > Am 11.07.2016 um 17:38 schrieb William A Rowe Jr <wr...@rowe-clan.net>:
> >
> > On Mon, Jul 11, 2016 at 5:25 AM, Stefan Eissing <st...@greenbytes.de> wrote:
> > In https://bz.apache.org/bugzilla/show_bug.cgi?id=59840 the issue crept up that StdEnvVars makes concurrent access to the SSL context when HTTP/2 is in place.
> >
> > The question is: how do we want to fix this?
> >
> > The general theme is that a module needs to perform some action on a connection, sees that it is a slave connection and accesses the master instead. Such a module needs to be aware that access to master may happen in parallel with other request.
> >
> > Should we let each such module care for it on its own or do we want to provide a way to access the master connection in a mutex protected way?
> >
> > Rather than a mutex, which would become an issue because all master access
> > to the context would require blocking, is there a way we can allow a module to
> > perform its (hopefully swift) processing on the master thread? Send a metadata
> > bucket to the master asking for a function to be invoked?
>
> Hmm, interesting idea. Of course, there would still be a mutex involved in handling such a query, but maybe it can be done more elegant than mutexing all accesses to structures everywhere.
>
> One way to look at this is that information retrieval from slave to master is something like a GET and that information injection into the master conn is something like a POST. So, effectively (and not necessarily in HTTP wire format), a slave connection would to a
>
> GET /ssl/StdEnvVars
>
> to retrieve the map. Which would, the first time, get added to the incoming queue on the master thread, be answered, slave thread awoken again and processing continues. This would be cached for future queries, so that slaves can access it without being put to sleep.
>
> We could use our famous HOOK infrastructure so that modules can register their own handlers for such information. We'd only need to nail down the general semantics and the data format for interchange. Something like a const apr_table_t would probably work. It would need to be readonly and allow concurrent use.
>
> The problem I encountered in mod_ftp was unbounded memory consumption in the control connection (similar schema to mod_http2 master/child)...
>
> In the mod_ftp case, it was login credentials. A unique pool off of the connection that could be cleared on a new login was necessary to keep us from consuming more and more pool on each login reattempt.
>
> Same will happen here, that apr_table_t of ssl envvars will need to survive until all requests running off of that list of connection strings are complete, and then released. Maybe a simple apr_atomic_inc32/dec32 would suffice for a ref count, based on a cleanup registered on the request pool? In any case, the request should use the result table it requested, and not some shared pointer in the conn_rec.
Yes. And my idea would be to keep the "cached" values at the slave connection. mod_http2 keeps a spare slave connection around, but surplus slaves are destroyed. That should eliminate a memory buildup on long-lived connections.
Re: concurrent ssl context access
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Tue, Jul 12, 2016 at 3:37 AM, Stefan Eissing <
stefan.eissing@greenbytes.de> wrote:
>
> > Am 11.07.2016 um 17:38 schrieb William A Rowe Jr <wr...@rowe-clan.net>:
> >
> > On Mon, Jul 11, 2016 at 5:25 AM, Stefan Eissing <
> stefan.eissing@greenbytes.de> wrote:
> > In https://bz.apache.org/bugzilla/show_bug.cgi?id=59840 the issue crept
> up that StdEnvVars makes concurrent access to the SSL context when HTTP/2
> is in place.
> >
> > The question is: how do we want to fix this?
> >
> > The general theme is that a module needs to perform some action on a
> connection, sees that it is a slave connection and accesses the master
> instead. Such a module needs to be aware that access to master may happen
> in parallel with other request.
> >
> > Should we let each such module care for it on its own or do we want to
> provide a way to access the master connection in a mutex protected way?
> >
> > Rather than a mutex, which would become an issue because all master
> access
> > to the context would require blocking, is there a way we can allow a
> module to
> > perform its (hopefully swift) processing on the master thread? Send a
> metadata
> > bucket to the master asking for a function to be invoked?
>
> Hmm, interesting idea. Of course, there would still be a mutex involved in
> handling such a query, but maybe it can be done more elegant than mutexing
> all accesses to structures everywhere.
>
> One way to look at this is that information retrieval from slave to master
> is something like a GET and that information injection into the master conn
> is something like a POST. So, effectively (and not necessarily in HTTP wire
> format), a slave connection would to a
>
> GET /ssl/StdEnvVars
>
> to retrieve the map. Which would, the first time, get added to the
> incoming queue on the master thread, be answered, slave thread awoken again
> and processing continues. This would be cached for future queries, so that
> slaves can access it without being put to sleep.
>
> We could use our famous HOOK infrastructure so that modules can register
> their own handlers for such information. We'd only need to nail down the
> general semantics and the data format for interchange. Something like a
> const apr_table_t would probably work. It would need to be readonly and
> allow concurrent use.
>
The problem I encountered in mod_ftp was unbounded memory consumption in
the control connection (similar schema to mod_http2 master/child)...
In the mod_ftp case, it was login credentials. A unique pool off of the
connection that could be cleared on a new login was necessary to keep us
from consuming more and more pool on each login reattempt.
Same will happen here, that apr_table_t of ssl envvars will need to survive
until all requests running off of that list of connection strings are
complete, and then released. Maybe a simple apr_atomic_inc32/dec32 would
suffice for a ref count, based on a cleanup registered on the request pool?
In any case, the request should use the result table it requested, and not
some shared pointer in the conn_rec.
Re: concurrent ssl context access
Posted by Stefan Eissing <st...@greenbytes.de>.
> Am 11.07.2016 um 17:38 schrieb William A Rowe Jr <wr...@rowe-clan.net>:
>
> On Mon, Jul 11, 2016 at 5:25 AM, Stefan Eissing <st...@greenbytes.de> wrote:
> In https://bz.apache.org/bugzilla/show_bug.cgi?id=59840 the issue crept up that StdEnvVars makes concurrent access to the SSL context when HTTP/2 is in place.
>
> The question is: how do we want to fix this?
>
> The general theme is that a module needs to perform some action on a connection, sees that it is a slave connection and accesses the master instead. Such a module needs to be aware that access to master may happen in parallel with other request.
>
> Should we let each such module care for it on its own or do we want to provide a way to access the master connection in a mutex protected way?
>
> Rather than a mutex, which would become an issue because all master access
> to the context would require blocking, is there a way we can allow a module to
> perform its (hopefully swift) processing on the master thread? Send a metadata
> bucket to the master asking for a function to be invoked?
Hmm, interesting idea. Of course, there would still be a mutex involved in handling such a query, but maybe it can be done more elegant than mutexing all accesses to structures everywhere.
One way to look at this is that information retrieval from slave to master is something like a GET and that information injection into the master conn is something like a POST. So, effectively (and not necessarily in HTTP wire format), a slave connection would to a
GET /ssl/StdEnvVars
to retrieve the map. Which would, the first time, get added to the incoming queue on the master thread, be answered, slave thread awoken again and processing continues. This would be cached for future queries, so that slaves can access it without being put to sleep.
We could use our famous HOOK infrastructure so that modules can register their own handlers for such information. We'd only need to nail down the general semantics and the data format for interchange. Something like a const apr_table_t would probably work. It would need to be readonly and allow concurrent use.
-Stefan
Re: concurrent ssl context access
Posted by Stefan Eissing <st...@greenbytes.de>.
> Am 11.07.2016 um 17:42 schrieb William A Rowe Jr <wr...@rowe-clan.net>:
> On Mon, Jul 11, 2016 at 10:38 AM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> On Mon, Jul 11, 2016 at 5:25 AM, Stefan Eissing <st...@greenbytes.de> wrote:
> In https://bz.apache.org/bugzilla/show_bug.cgi?id=59840 the issue crept up that StdEnvVars makes concurrent access to the SSL context when HTTP/2 is in place.
>
> The question is: how do we want to fix this?
>
> The general theme is that a module needs to perform some action on a connection, sees that it is a slave connection and accesses the master instead. Such a module needs to be aware that access to master may happen in parallel with other request.
>
> Should we let each such module care for it on its own or do we want to provide a way to access the master connection in a mutex protected way?
>
> Rather than a mutex, which would become an issue because all master access
> to the context would require blocking, is there a way we can allow a module to
> perform its (hopefully swift) processing on the master thread? Send a metadata
> bucket to the master asking for a function to be invoked?
>
> (Note that the ask of interrogating the ssl context for stdenvvars itself by the
> child request is a pretty sub-optimal solution. mod_ssl itself should gather
> and set this data aside once (and once again, upon renegotation) for our
> concurrent, un-mutexed reference in the child request.)
I think there should be env vars attached to conn_rec which get copied to the vars for a request_rec. If anyone has a good idea how to bring that into existence, I'm all for it.
Re: concurrent ssl context access
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Mon, Jul 11, 2016 at 10:38 AM, William A Rowe Jr <wr...@rowe-clan.net>
wrote:
> On Mon, Jul 11, 2016 at 5:25 AM, Stefan Eissing <
> stefan.eissing@greenbytes.de> wrote:
>
>> In https://bz.apache.org/bugzilla/show_bug.cgi?id=59840 the issue crept
>> up that StdEnvVars makes concurrent access to the SSL context when HTTP/2
>> is in place.
>>
>> The question is: how do we want to fix this?
>>
>> The general theme is that a module needs to perform some action on a
>> connection, sees that it is a slave connection and accesses the master
>> instead. Such a module needs to be aware that access to master may happen
>> in parallel with other request.
>>
>> Should we let each such module care for it on its own or do we want to
>> provide a way to access the master connection in a mutex protected way?
>>
>
> Rather than a mutex, which would become an issue because all master access
> to the context would require blocking, is there a way we can allow a
> module to
> perform its (hopefully swift) processing on the master thread? Send a
> metadata
> bucket to the master asking for a function to be invoked?
>
(Note that the ask of interrogating the ssl context for stdenvvars itself
by the
child request is a pretty sub-optimal solution. mod_ssl itself should
gather
and set this data aside once (and once again, upon renegotation) for our
concurrent, un-mutexed reference in the child request.)
Re: concurrent ssl context access
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Mon, Jul 11, 2016 at 5:25 AM, Stefan Eissing <
stefan.eissing@greenbytes.de> wrote:
> In https://bz.apache.org/bugzilla/show_bug.cgi?id=59840 the issue crept
> up that StdEnvVars makes concurrent access to the SSL context when HTTP/2
> is in place.
>
> The question is: how do we want to fix this?
>
> The general theme is that a module needs to perform some action on a
> connection, sees that it is a slave connection and accesses the master
> instead. Such a module needs to be aware that access to master may happen
> in parallel with other request.
>
> Should we let each such module care for it on its own or do we want to
> provide a way to access the master connection in a mutex protected way?
>
Rather than a mutex, which would become an issue because all master access
to the context would require blocking, is there a way we can allow a module
to
perform its (hopefully swift) processing on the master thread? Send a
metadata
bucket to the master asking for a function to be invoked?