You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Brian Akins <br...@turner.com> on 2007/02/02 15:11:16 UTC
mod_memcache??
I have a need to write a generic way to integrate apr_memcache into httpd.
Basically, I have several otehr modules taht use memcached as backend and want
to combine the "boring" stuff into a central place, ie configuration, stats,
etc. We talked a little on list about this a few months ago, but noone ever did
anything. Is anyone else interested in this? Has anyone did this?
Basically I was thinking there would be a single funtion:
apr_status_t ap_memcache_client(apr_memcache_t **mc)
which would simply give the "user" an client to use with normal apr_memcache
functions. The module could create the underlying mc at post_config.
Basically, mod_memcache could have this config:
MemCacheServer memcache1.turner.com:9020 min=8 smax=16 max=64 ttl=5
MemCacheServer memcache4.turner.com:9020 min=8 smax=16 max=64 ttl=5
MemCacheServer memcache10.turner.com:9020 min=8 smax=16 max=64 ttl=5
or whatever. This would end the config duplication between various modules.
This module could also add memcache stats to /server-status
Comments?
--
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies
Re: mod_memcache??
Posted by Nick Kew <ni...@webthing.com>.
On Wed, 07 Feb 2007 16:56:28 -0800
Paul Querna <ch...@force-elite.com> wrote:
> >> Comments?
> >
> > Sounds slightly analagous to mod_dbd: what I describe as a service
> > module in the book. Ending config duplication sounds like a Good
> > Thing, for the same reasons as DBD.
> >
> > Couple of thoughts:
> >
> > * What's the system overhead of memcache? Is it similar to DBM?
>
> Not sure what you mean... it mostly means lots of extra TCP sockets
> open all the time.
I mainly meant "so, we won't be repeating those mod_dbd design
decisions that were dictated by the fact that SQL queries are
inherently rather expensive". Or might we? With DBM, I could
answer that one authoritatively:-)
> > * I'm not really familiar with memcache, but the semantics look
> > somewhat like DBM. Would there be any mileage in supporting
> > a DBM store in a unified functional API?
>
> It supports more things; Like timeouts and some other things like
> increment/decrement counters; It could be done on top of DBM, but I'm
> not really interested in it.
"On top of"? As in the dbm+rpc implementation I hacked up many
years ago? What I had in mind was just a common frontend.
Anyway, I'm not volunteering to do the hacking, so I'll shut up:-)
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/
Re: mod_memcache??
Posted by Paul Querna <ch...@force-elite.com>.
Nick Kew wrote:
> On Fri, 02 Feb 2007 09:11:16 -0500
> Brian Akins <br...@turner.com> wrote:
>
>> Basically, mod_memcache could have this config:
>>
>> MemCacheServer memcache1.turner.com:9020 min=8 smax=16 max=64 ttl=5
>> MemCacheServer memcache4.turner.com:9020 min=8 smax=16 max=64 ttl=5
>> MemCacheServer memcache10.turner.com:9020 min=8 smax=16 max=64 ttl=5
>>
>> or whatever. This would end the config duplication between various
>> modules. This module could also add memcache stats to /server-status
>>
>> Comments?
>
> Sounds slightly analagous to mod_dbd: what I describe as a service
> module in the book. Ending config duplication sounds like a Good
> Thing, for the same reasons as DBD.
>
> Couple of thoughts:
>
> * What's the system overhead of memcache? Is it similar to DBM?
Not sure what you mean... it mostly means lots of extra TCP sockets open
all the time.
> * I'm not really familiar with memcache, but the semantics look
> somewhat like DBM. Would there be any mileage in supporting
> a DBM store in a unified functional API?
It supports more things; Like timeouts and some other things like
increment/decrement counters; It could be done on top of DBM, but I'm
not really interested in it.
> * Is it inherently thread-safe, or will you be needing a reslist
> or similar solution?
>
apr_memcache internally uses a reslist, and itself is 'thread' safe once
configured.
-Paul
Re: protocol module ?
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
All hooks may see any connection (likewise, with other hooks). The trick
is how to 'go away' and ignore/decline handling it.
mod_ftp looks for a server's FTP On directive setting in the server host
context to decide to pick up and handle the connection. Since vhosts
can be configured for VirtualHost _default_:port or VirtualHost IP:port,
this is pretty trivial to unwind if the user places their protocol
related directive in the proper context. See mod_echo for an example.
Gregory Nicholls wrote:
> I've been looking at mod-ftpd and mod-pop3 with a view to writing a
> protocol module. What I haven't been able to work out from reading
> these, is how a particular protocol module 'registers' its interest in
> specific connections.
> eg. Assume there's 2 protocols being used and we have Listen 8080 http
> and Listen 1234 foo. I use ap_hook_process_connection to insert mod_foo
> into the chain. So far so good but I only want to process connections
> coming on port 1234. Neither mod-ftpd or mod-pop3 seem to have any code
> that examines the connection with a view to declining those they can't
> process. So I'm guessing that there's some mechanism for assigning
> protocols/ports to a specific module. I just don't know how to do it.
> Can some kind soul point out what I've missed please ?
> Thanks,
> Gregory Nicholls
>
> .
>
Re: protocol module ?
Posted by Eli Marmor <ma...@netmask.it>.
Gregory Nicholls wrote:
>
> I've been looking at mod-ftpd and mod-pop3 with a view to writing a
> protocol module. What I haven't been able to work out from reading
> these, is how a particular protocol module 'registers' its interest in
> specific connections.
> eg. Assume there's 2 protocols being used and we have Listen 8080 http
> and Listen 1234 foo. I use ap_hook_process_connection to insert mod_foo
> into the chain. So far so good but I only want to process connections
> coming on port 1234. Neither mod-ftpd or mod-pop3 seem to have any code
> that examines the connection with a view to declining those they can't
> process. So I'm guessing that there's some mechanism for assigning
> protocols/ports to a specific module. I just don't know how to do it.
> Can some kind soul point out what I've missed please ?
> Thanks,
> Gregory Nicholls
I don't know if it will help you, but Ryan Bloom dedicated a chapter
for that in his book ("Using Apache to Serve Different Protocols", pp.
383-401).
In addition, mod_ftpd and mod_pop3 are not the only non-httpd protocol
modules, and there are some else, such as mod_smtpd. I wish there was
a mod_named (or mod_dns or mod_bind, I don't care). One day, maybe I'll
write one...
--
Eli Marmor
Netmask (El-Mar) Internet Technologies Ltd.
__________________________________________________________
Tel.: +972-9-766-1020 8 Yad-Harutzim St.
Fax.: +972-9-766-1314 P.O.B. 7004
Mobile: +972-50-5237338 Kfar-Saba 44641, Israel
protocol module ?
Posted by Gregory Nicholls <gn...@bellsouth.net>.
I've been looking at mod-ftpd and mod-pop3 with a view to writing a
protocol module. What I haven't been able to work out from reading
these, is how a particular protocol module 'registers' its interest in
specific connections.
eg. Assume there's 2 protocols being used and we have Listen 8080 http
and Listen 1234 foo. I use ap_hook_process_connection to insert mod_foo
into the chain. So far so good but I only want to process connections
coming on port 1234. Neither mod-ftpd or mod-pop3 seem to have any code
that examines the connection with a view to declining those they can't
process. So I'm guessing that there's some mechanism for assigning
protocols/ports to a specific module. I just don't know how to do it.
Can some kind soul point out what I've missed please ?
Thanks,
Gregory Nicholls
Re: mod_memcache??
Posted by Nick Kew <ni...@webthing.com>.
On Fri, 02 Feb 2007 09:11:16 -0500
Brian Akins <br...@turner.com> wrote:
> Basically, mod_memcache could have this config:
>
> MemCacheServer memcache1.turner.com:9020 min=8 smax=16 max=64 ttl=5
> MemCacheServer memcache4.turner.com:9020 min=8 smax=16 max=64 ttl=5
> MemCacheServer memcache10.turner.com:9020 min=8 smax=16 max=64 ttl=5
>
> or whatever. This would end the config duplication between various
> modules. This module could also add memcache stats to /server-status
>
> Comments?
Sounds slightly analagous to mod_dbd: what I describe as a service
module in the book. Ending config duplication sounds like a Good
Thing, for the same reasons as DBD.
Couple of thoughts:
* What's the system overhead of memcache? Is it similar to DBM?
* I'm not really familiar with memcache, but the semantics look
somewhat like DBM. Would there be any mileage in supporting
a DBM store in a unified functional API?
* Is it inherently thread-safe, or will you be needing a reslist
or similar solution?
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/
Re: mod_memcache??
Posted by josh rotenberg <jo...@gmail.com>.
Hi,
So I've uploaded a 0.1.0 version of this. Definitely still pretty
simple and rough but it seems to be working ok. I've got a few ideas
for making the configuration a little nicer, and I'll probably try to
implement them soon, but if you (Frank, et al) wanted to take a look
and make suggestions or patches, I'm happy to listen and/or apply.
Thanks,
Josh
On 7/1/07, josh rotenberg <jo...@gmail.com> wrote:
> Hi,
>
> I've got > 1 module projects using apr_memcache now, so this would
> actually come in handy for me. I've started some development, and
> created a google project for it at
> http://code.google.com/p/modmemcache/
>
> I just checked in some untested, non functioning stub files to get the
> ball rolling. If anyone wants to be added to the project and help out,
> just let me know. At this point I plan to implement as spec'ed here by
> Brian:
>
> http://mail-archives.apache.org/mod_mbox/httpd-dev/200702.mbox/%3c45C34684.7050403@turner.com%3e
>
> i.e. use the same config format and, for now, export the single
> function that gives access to an already initialized apr_memcache_t.
> Ideas for any other functionality are greatly appreciated.
>
> Thanks,
>
> Josh
>
> On 7/1/07, Akins, Brian <Br...@turner.com> wrote:
> > On 7/1/07 10:11 AM, "Frank" <fr...@x09.de> wrote:
> >
> > >
> > > I just wonder what has happen to this good idea? Did you start
> > > implementing it? (Today I was thinking about implementing this, coz' I
> > > need it....)
> >
> > Never had the time. Project at work went a different direction.
> >
> >
> > --
> > Brian Akins
> > Chief Operations Engineer
> > Turner Digital Media Technologies
> >
>
Re: mod_memcache??
Posted by josh rotenberg <jo...@gmail.com>.
Hi,
I've got > 1 module projects using apr_memcache now, so this would
actually come in handy for me. I've started some development, and
created a google project for it at
http://code.google.com/p/modmemcache/
I just checked in some untested, non functioning stub files to get the
ball rolling. If anyone wants to be added to the project and help out,
just let me know. At this point I plan to implement as spec'ed here by
Brian:
http://mail-archives.apache.org/mod_mbox/httpd-dev/200702.mbox/%3c45C34684.7050403@turner.com%3e
i.e. use the same config format and, for now, export the single
function that gives access to an already initialized apr_memcache_t.
Ideas for any other functionality are greatly appreciated.
Thanks,
Josh
On 7/1/07, Akins, Brian <Br...@turner.com> wrote:
> On 7/1/07 10:11 AM, "Frank" <fr...@x09.de> wrote:
>
> >
> > I just wonder what has happen to this good idea? Did you start
> > implementing it? (Today I was thinking about implementing this, coz' I
> > need it....)
>
> Never had the time. Project at work went a different direction.
>
>
> --
> Brian Akins
> Chief Operations Engineer
> Turner Digital Media Technologies
>
Re: mod_memcache??
Posted by "Akins, Brian" <Br...@turner.com>.
On 7/1/07 10:11 AM, "Frank" <fr...@x09.de> wrote:
>
> I just wonder what has happen to this good idea? Did you start
> implementing it? (Today I was thinking about implementing this, coz' I
> need it....)
Never had the time. Project at work went a different direction.
--
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies
Re: mod_memcache??
Posted by Frank <fr...@x09.de>.
Brian Akins wrote:
> I have a need to write a generic way to integrate apr_memcache into
> httpd. Basically, I have several otehr modules taht use memcached as
> [...]
> Basically, mod_memcache could have this config:
>
> MemCacheServer memcache1.turner.com:9020 min=8 smax=16 max=64 ttl=5
> MemCacheServer memcache4.turner.com:9020 min=8 smax=16 max=64 ttl=5
> MemCacheServer memcache10.turner.com:9020 min=8 smax=16 max=64 ttl=5
> [...]
Hi Brian,
I just wonder what has happen to this good idea? Did you start
implementing it? (Today I was thinking about implementing this, coz' I
need it....)
Or do you still think about an abstraction level for merging modules
like mod_memcache and mod_dbd? :-)
Frank
Re: [info@ch2o.info] Re: mod_memcache??
Posted by Mathieu CARBONNEAUX <in...@ch2o.info>.
interesting idea !!!
i use memcache (with libmemcache) in my from auth module (http://authmemcookie.sourceforge.net/)!
my next step is to try to use apr_memcache in place of libmemcache... (by my primary goal is to make posible to use my module on old 2.0 apache version that user older apr&apr-util than 0.9.5...)
but your addition while simplify the use of apr_memcache!
if i anderstand your idea is to developpe module to make apr_memcache connection pool like mod_dbi_pool (http://www.outoforder.cc/projects/apache/mod_dbi_pool/) ?
Regards,
Mathieu
_____
From: Garrett Rooney [mailto:rooneg@electricjellyfish.net]
To: dev@httpd.apache.org
Sent: Mon, 05 Feb 2007 16:04:03 +0100
Subject: [info@ch2o.info] Re: mod_memcache??
On 2/2/07, Brian Akins <br...@turner.com> wrote:
> I have a need to write a generic way to integrate apr_memcache into httpd.
> Basically, I have several otehr modules taht use memcached as backend and want
> to combine the "boring" stuff into a central place, ie configuration, stats,
> etc. We talked a little on list about this a few months ago, but noone ever did
> anything. Is anyone else interested in this? Has anyone did this?
>
> Basically I was thinking there would be a single funtion:
>
> apr_status_t ap_memcache_client(apr_memcache_t **mc)
>
> which would simply give the "user" an client to use with normal apr_memcache
> functions. The module could create the underlying mc at post_config.
>
> Basically, mod_memcache could have this config:
>
> MemCacheServer memcache1.turner.com:9020 min=8 smax=16 max=64 ttl=5
> MemCacheServer memcache4.turner.com:9020 min=8 smax=16 max=64 ttl=5
> MemCacheServer memcache10.turner.com:9020 min=8 smax=16 max=64 ttl=5
>
> or whatever. This would end the config duplication between various modules.
> This module could also add memcache stats to /server-status
>
> Comments?
Seems useful to me.
-garrett
Re: mod_memcache??
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 2/2/07, Brian Akins <br...@turner.com> wrote:
> I have a need to write a generic way to integrate apr_memcache into httpd.
> Basically, I have several otehr modules taht use memcached as backend and want
> to combine the "boring" stuff into a central place, ie configuration, stats,
> etc. We talked a little on list about this a few months ago, but noone ever did
> anything. Is anyone else interested in this? Has anyone did this?
>
> Basically I was thinking there would be a single funtion:
>
> apr_status_t ap_memcache_client(apr_memcache_t **mc)
>
> which would simply give the "user" an client to use with normal apr_memcache
> functions. The module could create the underlying mc at post_config.
>
> Basically, mod_memcache could have this config:
>
> MemCacheServer memcache1.turner.com:9020 min=8 smax=16 max=64 ttl=5
> MemCacheServer memcache4.turner.com:9020 min=8 smax=16 max=64 ttl=5
> MemCacheServer memcache10.turner.com:9020 min=8 smax=16 max=64 ttl=5
>
> or whatever. This would end the config duplication between various modules.
> This module could also add memcache stats to /server-status
>
> Comments?
Seems useful to me.
-garrett