You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modules-dev@httpd.apache.org by Kevac Marko <ma...@kevac.org> on 2009/03/05 15:57:25 UTC

mod_dbd analogue for memcached

Hello

I've created mod_dbd analogue for memcached (named connection pools to
memcached servers, see also
http://www.mail-archive.com/dev@httpd.apache.org/msg42922.html).

It's now based on libmemcached (http://tangent.org/552/libmemcached.html).

The question I want to ask is should I base my module on apr_memcache
or not? Is apr_memcache mature enough? Whether is is used by someone?

Thanks.

-- 
Marko Kevac

Re: mod_dbd analogue for memcached

Posted by Jérôme Renard <jr...@ez.no>.
Hi Marko,

Kevac Marko a écrit :
> Hello
> 
> I've created mod_dbd analogue for memcached (named connection pools to
> memcached servers, see also
> http://www.mail-archive.com/dev@httpd.apache.org/msg42922.html).
> 
> It's now based on libmemcached (http://tangent.org/552/libmemcached.html).
> 
> The question I want to ask is should I base my module on apr_memcache
> or not? Is apr_memcache mature enough? Whether is is used by someone?

I use it for the following module :
http://code.google.com/p/modmemcachedinclude/

Best Regards.

-- 
Jérôme :)

Re: mod_dbd analogue for memcached

Posted by "M. Brian Akins" <br...@akins.org>.
On Mar 6, 2009, at 1:25 AM, Kevac Marko wrote:
>
> And after that you have to restart apache? It's not enough dynamic  
> for us.
>

graceful restart.

Re: mod_dbd analogue for memcached

Posted by Kevac Marko <ma...@kevac.org>.
On Fri, Mar 6, 2009 at 2:02 AM, M. Brian Akins <br...@akins.org> wrote:
>
> Will you implement a "default" pool like mod_dbd does?

Yes

>> This feature enables us to create dynamic, not static servers list.
>> For high availability clusters for example.
>>
>
> Sounds interesting, I suppose. I just generate my configs from templates and
> they generate the correct server list based on whatever

And after that you have to restart apache? It's not enough dynamic for us.

We are dynamically changing memcached servers list without Apache
restart. Based on information provided by small network daemon via
shared memory (you can use SNMP here as well).

It allows us to dynamically and on the fly add new cluster nodes with
memcached instances or remove ones that we don't need.

-- 
Marko Kevac

Re: mod_dbd analogue for memcached

Posted by "M. Brian Akins" <br...@akins.org>.
On Mar 5, 2009, at 4:35 PM, Kevac Marko wrote:
> ise.
>
> mod_memcache, if we are talking about
> http://code.google.com/p/modmemcache/, is too simple.
>

Works fine for me :)

> I need multiple name pools to multiple servers. Something like
>


Will you implement a "default" pool like mod_dbd does?

> This feature enables us to create dynamic, not static servers list.
> For high availability clusters for example.
>

Sounds interesting, I suppose. I just generate my configs from  
templates and they generate the correct server list based on whatever

Re: mod_dbd analogue for memcached

Posted by Kevac Marko <ma...@kevac.org>.
On Thu, Mar 5, 2009 at 6:53 PM, Brian Akins <br...@akins.org> wrote:
> Is this based on the existing mod_memcache?

No. It is based on mod_dbd. New one
(http://www.mail-archive.com/dev@httpd.apache.org/msg42922.html) to be
precise.

mod_memcache, if we are talking about
http://code.google.com/p/modmemcache/, is too simple.

I need multiple name pools to multiple servers. Something like

<MemcachePool "name1">
    Server 192.168.0.1:11211
    Server 192.168.0.2:11211
    Server 192.168.0.3:11211
    Server 192.168.0.4:11211
    Server 192.168.0.5:11211
    Other options...
</MemcachePool>

<MemcachePool "name2">
    Server 192.168.0.6:11211
    Server 192.168.0.7:11211
    Server 192.168.0.8:11211
    Server 192.168.0.9:11211
    Server 192.168.0.10:11211
    Other options...
</MemcachePool>

Also I will implement hook in module. Hooked function can filter servers list.

For example (in python pseudo code):

def dead_or_alive(servers_list):
    new_list = []
    for server in servers_list:
        if server is in some list in shared memory,
            it is alive and it is not overloaded then:
            new_list.add(server)

    return new_list

If returned list differs from saved one, than we recreate connections
(maybe something smarter).

This feature enables us to create dynamic, not static servers list.
For high availability clusters for example.

Alright, i will ty to base it on apr_memcache then.

-- 
Marko Kevac

Re: mod_dbd analogue for memcached

Posted by Kevac Marko <ma...@kevac.org>.
On Tue, Mar 10, 2009 at 8:49 PM, josh rotenberg <jo...@gmail.com> wrote:
> I'm just catching up, but I'd be happy to add some of this
> functionality to mod_memcache.

Great. I think I'll send patch on this week.

mod_memcache is not in httpd, so, probably, we should continue our
talk somewhere else?

Do you have plans to push mod_memcache to httpd-trunk?

-- 
Marko Kevac

Re: mod_dbd analogue for memcached

Posted by josh rotenberg <jo...@gmail.com>.
Hiya,

> For our needs, mod_memcache (http://code.google.com/p/modmemcache/)
> will need one major change: memcache groups.
>
> <MemcacheGroup group01>
>    MemcacheServer localhost:1234
>    MemcacheServer localhost:1235
> </MemcacheGroup>
>
> <MemcacheGroup group01>
>    MemcacheServer localhost:1234
>    MemcacheServer localhost:1235
> </MemcacheGroup>

I'm just catching up, but I'd be happy to add some of this
functionality to mod_memcache.

Josh

Re: mod_dbd analogue for memcached

Posted by Kevac Marko <ma...@kevac.org>.
On Thu, Mar 5, 2009 at 6:53 PM, Brian Akins <br...@akins.org> wrote:
> Is this based on the existing mod_memcache?

After some observation of apr_memcache I have realized that mod_dbd
pools are not needed here, because apr_memcache have own pools to each
memcache server. So it can be based on existing mod_memcache.

For our needs, mod_memcache (http://code.google.com/p/modmemcache/)
will need one major change: memcache groups.

<MemcacheGroup group01>
    MemcacheServer localhost:1234
    MemcacheServer localhost:1235
</MemcacheGroup>

<MemcacheGroup group01>
    MemcacheServer localhost:1234
    MemcacheServer localhost:1235
</MemcacheGroup>

And memcache API:

ap_memcache_client(server_rec *s)
ap_memcache_client_group(server_rec *s, char *group_name)

As i have already mentioned, we need dynamic memcache servers within
groups. That can be implemented as separated module which can take
memcache structure from mod_memcache and enable/disable/add/remove
needed servers according to some rules.

-- 
Marko Kevac

Re: mod_dbd analogue for memcached

Posted by Brian Akins <br...@akins.org>.
On 3/5/09 9:57 AM, "Kevac Marko" <ma...@kevac.org> wrote:

> The question I want to ask is should I base my module on apr_memcache
> or not? Is apr_memcache mature enough? Whether is is used by someone?

Yes. Yes. Yes.

Is this based on the existing mod_memcache?

--Brian