You are viewing a plain text version of this content. The canonical link for it is here.
Posted to docs@httpd.apache.org by Igor Galić <i....@brainsware.org> on 2010/12/29 02:25:10 UTC
Inspiration for mod_lua
Hey folks,
I'm looking for some inspiration on how to make good use of
mod_lua. Those familiar with its documentation, might find
it a little bit lacking in this regard.
I've got an elaborate project plotted out, which I'll implement
real soon now, but what I'm chiefly looking for is what mod_lua
has promised us: A cure for mod_rewrite.
Please share your particularly ugly, involved, unaesthetic or
otherwise /wrong/ solutions done with mod_rewrite because it
was the only hammer available for the screws at that time ;)
To extend our gallery:
http://wiki.apache.org/httpd/WhenNotToUseRewrite
on one hand, and on the other to find inspiration to solve the
(more involved?) problems - if possible or sensible - in mod_lua.
So long,
i
--
Igor Galić
Tel: +43 (0) 664 886 22 883
Mail: i.galic@brainsware.org
URL: http://brainsware.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: docs-unsubscribe@httpd.apache.org
For additional commands, e-mail: docs-help@httpd.apache.org
Re: Inspiration for mod_lua
Posted by Brian McCallister <br...@skife.org>.
On Fri, Dec 31, 2010 at 12:48 AM, William A. Rowe Jr.
<wr...@rowe-clan.net> wrote:
> On 12/30/2010 8:09 PM, Joe Schaefer wrote:
>>
>> I mean take the CMS I just wrote with mod_perl. It's 2K LOC, uses a custom
>> map_to_storage handler, and defers a good chunk of its (sub)requests to httpd
>> for things like autoindex and negotiation and plain-old file serving, and
>> forwards
>> a user's basic auth creds to the subversion server. That isn't remotely
>> feasible
>> with mod_fcgi, I'd have to reimplement all that functionality my app, which is
>> a waste.
>
> You are right, for content fcgid would be fine, but for the rest we'd hope that
> mod_lua is much less of a memory hog, and can accomplish those same things.
Honestly, I don't expect mod_lua to see heavy use for content
generation (will be happy for that to prove wrong), but python and
ruby rule that roost right now, and external process proxying (fcgi,
mod_proxy, mod_wsgi, whatever) all work great there. My personal
target is everything *else* that you can hook :-)
>
Re: Inspiration for mod_lua
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 12/30/2010 8:09 PM, Joe Schaefer wrote:
>
> I mean take the CMS I just wrote with mod_perl. It's 2K LOC, uses a custom
> map_to_storage handler, and defers a good chunk of its (sub)requests to httpd
> for things like autoindex and negotiation and plain-old file serving, and
> forwards
> a user's basic auth creds to the subversion server. That isn't remotely
> feasible
> with mod_fcgi, I'd have to reimplement all that functionality my app, which is
> a waste.
You are right, for content fcgid would be fine, but for the rest we'd hope that
mod_lua is much less of a memory hog, and can accomplish those same things.
Re: Inspiration for mod_lua
Posted by Joe Schaefer <jo...@yahoo.com>.
To get back to lua, mod_lua should IMO expose as much of the server
API as it can, so people who want to script the server's behavior
in a high-concurrency environment can do so. It should definitely
compete with mod_rewrite, and if successful could supplant it.
But it shouldn't try to "compete" with things like rails or even mod_perl,
because those communities have a massive set of complementary modules
people will use (most of which probably aren't even interpreter thread-
safe). It's a different set of tradeoffs between convenience and power.
----- Original Message ----
> From: Joe Schaefer <jo...@yahoo.com>
> To: dev@httpd.apache.org
> Sent: Thu, December 30, 2010 9:09:32 PM
> Subject: Re: Inspiration for mod_lua
>
> ----- Original Message ----
>
> > From: William A. Rowe Jr. <wr...@rowe-clan.net>
> > To: dev@httpd.apache.org
> > Sent: Thu, December 30, 2010 8:48:51 PM
> > Subject: Re: Inspiration for mod_lua
> >
> > On 12/30/2010 6:52 PM, Joe Schaefer wrote:
> > >
> > > Blech. The upside to mod_perl is that you get the rest of the server
for
> > > free. mod_fcgid (or even mod_wsgi) is the same old crappy impoverished
> > > CGI interface.
> >
> > Hmmm, fork for exec is that insignificant? Not according to any data I've
> >seen.
> >
> > I agree mod_perl has some killer perks, but really hoping that mod_lua
will
> > replace a good number of those special cases. As far as
>handlers/endpoints,
> > fcgid significantly displaces CGI in performance, although yes, it's
> identical
> > in its limitations.
>
> I wasn't referring to performance, but *nothing* other than the main websites
> would be impacted performance wise if they were all running mod_perl +
>prefork.
> I was referring to the fact that mod_perl exposes pretty much the entire
> webserver
> to perl, even though only a small number of people are able to use that
power
> effectively. Most people nowadays want a fully-boxed framework like rails.
>
> I mean take the CMS I just wrote with mod_perl. It's 2K LOC, uses a custom
> map_to_storage handler, and defers a good chunk of its (sub)requests to httpd
> for things like autoindex and negotiation and plain-old file serving, and
> forwards
> a user's basic auth creds to the subversion server. That isn't remotely
> feasible
> with mod_fcgi, I'd have to reimplement all that functionality my app, which
is
> a waste.
>
>
>
>
Re: Inspiration for mod_lua
Posted by Joe Schaefer <jo...@yahoo.com>.
----- Original Message ----
> From: William A. Rowe Jr. <wr...@rowe-clan.net>
> To: dev@httpd.apache.org
> Sent: Thu, December 30, 2010 8:48:51 PM
> Subject: Re: Inspiration for mod_lua
>
> On 12/30/2010 6:52 PM, Joe Schaefer wrote:
> >
> > Blech. The upside to mod_perl is that you get the rest of the server for
> > free. mod_fcgid (or even mod_wsgi) is the same old crappy impoverished
> > CGI interface.
>
> Hmmm, fork for exec is that insignificant? Not according to any data I've
>seen.
>
> I agree mod_perl has some killer perks, but really hoping that mod_lua will
> replace a good number of those special cases. As far as handlers/endpoints,
> fcgid significantly displaces CGI in performance, although yes, it's
identical
> in its limitations.
I wasn't referring to performance, but *nothing* other than the main websites
would be impacted performance wise if they were all running mod_perl + prefork.
I was referring to the fact that mod_perl exposes pretty much the entire
webserver
to perl, even though only a small number of people are able to use that power
effectively. Most people nowadays want a fully-boxed framework like rails.
I mean take the CMS I just wrote with mod_perl. It's 2K LOC, uses a custom
map_to_storage handler, and defers a good chunk of its (sub)requests to httpd
for things like autoindex and negotiation and plain-old file serving, and
forwards
a user's basic auth creds to the subversion server. That isn't remotely
feasible
with mod_fcgi, I'd have to reimplement all that functionality my app, which is
a waste.
Re: Inspiration for mod_lua
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 12/30/2010 6:52 PM, Joe Schaefer wrote:
>
> Blech. The upside to mod_perl is that you get the rest of the server for
> free. mod_fcgid (or even mod_wsgi) is the same old crappy impoverished
> CGI interface.
Hmmm, fork for exec is that insignificant? Not according to any data I've seen.
I agree mod_perl has some killer perks, but really hoping that mod_lua will
replace a good number of those special cases. As far as handlers/endpoints,
fcgid significantly displaces CGI in performance, although yes, it's identical
in its limitations.
Re: Inspiration for mod_lua
Posted by Joe Schaefer <jo...@yahoo.com>.
----- Original Message ----
> From: William A. Rowe Jr. <wr...@rowe-clan.net>
> To: dev@httpd.apache.org
> Sent: Thu, December 30, 2010 7:45:54 PM
> Subject: Re: Inspiration for mod_lua
>
> On 12/30/2010 6:08 PM, Graham Dumpleton wrote:
> >
> > The problem with mod_python was that it was poorly implemented. If you
> > were start from scratch and do it over, it would be possible to make
> > it much more light weight. The mod_wsgi module has shown this can be
> > the case.
>
> The best solution of perl/python/JavaVM/.NET is and will always be out of
> memory workers scaled for the anticipated amount of app traffic in some
> functionally useful scope (e.g. 8 fast dispatching workers on 4-way SMP).
> The machine doesn't bog, the responses appear more 'zippy', etc. Large
> response bodies dodge that worker pool.
>
> In other words, mod_fcgid. :)
Blech. The upside to mod_perl is that you get the rest of the server for
free. mod_fcgid (or even mod_wsgi) is the same old crappy impoverished
CGI interface.
Re: Inspiration for mod_lua
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 12/30/2010 6:08 PM, Graham Dumpleton wrote:
>
> The problem with mod_python was that it was poorly implemented. If you
> were start from scratch and do it over, it would be possible to make
> it much more light weight. The mod_wsgi module has shown this can be
> the case.
The best solution of perl/python/JavaVM/.NET is and will always be out of
memory workers scaled for the anticipated amount of app traffic in some
functionally useful scope (e.g. 8 fast dispatching workers on 4-way SMP).
The machine doesn't bog, the responses appear more 'zippy', etc. Large
response bodies dodge that worker pool.
In other words, mod_fcgid. :)
Re: Inspiration for mod_lua
Posted by Graham Dumpleton <gr...@gmail.com>.
On 31 December 2010 10:56, William A. Rowe Jr. <wr...@rowe-clan.net> wrote:
> On 12/30/2010 3:25 PM, Graham Dumpleton wrote:
>> On 31 December 2010 07:37, Brian McCallister <br...@skife.org> wrote:
>>> 2010/12/28 Igor Galić <i....@brainsware.org>:
>>>> Hey folks,
>>>>
>>>> I'm looking for some inspiration on how to make good use of
>>>> mod_lua. Those familiar with its documentation, might find
>>>> it a little bit lacking in this regard.
>>>
>>> My original aim (and what I still use mod_wombat for) is various small
>>> modules I don't want to be bothered using C for, but which need to run
>>> in a threaded MPM (making mod_python/mod_perl not viable options).
>>
>> Ignoring the fact that mod_python is now dead, there was never a
>> restriction on using mod_python in a threaded MPM.
>
> Nor for properly deployed mod_perl, but either is far more heavyweight than
> lua. And when you multiply interpreter contexts across worker threads, both
> mod_perl and mod_python suffer huge bloat. I'm hoping we see much different
> results with lua as the 'defacto' scripting engine.
The problem with mod_python was that it was poorly implemented. If you
were start from scratch and do it over, it would be possible to make
it much more light weight. The mod_wsgi module has shown this can be
the case. Problem is that mod_python's failings have resulted in this
overall perception that embedding Python inside of an Apache module is
bad, when it doesn't need to be. The problems with mod_python weren't
made any better through poor Python installations which didn't provide
a shared Python library. End result was that library was linked
statically and each process ended up with its own copy because of
forced address relocations, thus contributing to the perception of
memory bloat.
Anyway, all too late now as the perception that Python as scripting
language inside of Apache is bad is too prevalent and people continue
to propagate this even though in reality they are really misinformed.
:-(
Graham
Re: Inspiration for mod_lua
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On 12/30/2010 3:25 PM, Graham Dumpleton wrote:
> On 31 December 2010 07:37, Brian McCallister <br...@skife.org> wrote:
>> 2010/12/28 Igor Galić <i....@brainsware.org>:
>>> Hey folks,
>>>
>>> I'm looking for some inspiration on how to make good use of
>>> mod_lua. Those familiar with its documentation, might find
>>> it a little bit lacking in this regard.
>>
>> My original aim (and what I still use mod_wombat for) is various small
>> modules I don't want to be bothered using C for, but which need to run
>> in a threaded MPM (making mod_python/mod_perl not viable options).
>
> Ignoring the fact that mod_python is now dead, there was never a
> restriction on using mod_python in a threaded MPM.
Nor for properly deployed mod_perl, but either is far more heavyweight than
lua. And when you multiply interpreter contexts across worker threads, both
mod_perl and mod_python suffer huge bloat. I'm hoping we see much different
results with lua as the 'defacto' scripting engine.
Re: Inspiration for mod_lua
Posted by Graham Dumpleton <gr...@gmail.com>.
On 31 December 2010 07:37, Brian McCallister <br...@skife.org> wrote:
> 2010/12/28 Igor Galić <i....@brainsware.org>:
>> Hey folks,
>>
>> I'm looking for some inspiration on how to make good use of
>> mod_lua. Those familiar with its documentation, might find
>> it a little bit lacking in this regard.
>
> My original aim (and what I still use mod_wombat for) is various small
> modules I don't want to be bothered using C for, but which need to run
> in a threaded MPM (making mod_python/mod_perl not viable options).
Ignoring the fact that mod_python is now dead, there was never a
restriction on using mod_python in a threaded MPM.
Graham
> Auth against a remote service, interaction with shared memory for some
> other things on the box, graylisting access control, etc.
>
> I know it has also been used in the wild for browser sniffing for
> mobile devices, which means updating things frequently.
>
> A very nice use case I have seen is basically using it as the initial
> impl for things that will eventually become conventional C modules,
> once the functionality is understood and stabilises enough to make it
> worth it.
>
> -Brian
>
> -Brian
>
>>
>> I've got an elaborate project plotted out, which I'll implement
>> real soon now, but what I'm chiefly looking for is what mod_lua
>> has promised us: A cure for mod_rewrite.
>>
>> Please share your particularly ugly, involved, unaesthetic or
>> otherwise /wrong/ solutions done with mod_rewrite because it
>> was the only hammer available for the screws at that time ;)
>>
>> To extend our gallery:
>> http://wiki.apache.org/httpd/WhenNotToUseRewrite
>> on one hand, and on the other to find inspiration to solve the
>> (more involved?) problems - if possible or sensible - in mod_lua.
>>
>> So long,
>> i
>>
>> --
>> Igor Galić
>>
>> Tel: +43 (0) 664 886 22 883
>> Mail: i.galic@brainsware.org
>> URL: http://brainsware.org/
>>
>
Re: Inspiration for mod_lua
Posted by Brian McCallister <br...@skife.org>.
2010/12/28 Igor Galić <i....@brainsware.org>:
> Hey folks,
>
> I'm looking for some inspiration on how to make good use of
> mod_lua. Those familiar with its documentation, might find
> it a little bit lacking in this regard.
My original aim (and what I still use mod_wombat for) is various small
modules I don't want to be bothered using C for, but which need to run
in a threaded MPM (making mod_python/mod_perl not viable options).
Auth against a remote service, interaction with shared memory for some
other things on the box, graylisting access control, etc.
I know it has also been used in the wild for browser sniffing for
mobile devices, which means updating things frequently.
A very nice use case I have seen is basically using it as the initial
impl for things that will eventually become conventional C modules,
once the functionality is understood and stabilises enough to make it
worth it.
-Brian
-Brian
>
> I've got an elaborate project plotted out, which I'll implement
> real soon now, but what I'm chiefly looking for is what mod_lua
> has promised us: A cure for mod_rewrite.
>
> Please share your particularly ugly, involved, unaesthetic or
> otherwise /wrong/ solutions done with mod_rewrite because it
> was the only hammer available for the screws at that time ;)
>
> To extend our gallery:
> http://wiki.apache.org/httpd/WhenNotToUseRewrite
> on one hand, and on the other to find inspiration to solve the
> (more involved?) problems - if possible or sensible - in mod_lua.
>
> So long,
> i
>
> --
> Igor Galić
>
> Tel: +43 (0) 664 886 22 883
> Mail: i.galic@brainsware.org
> URL: http://brainsware.org/
>
Re: Inspiration for mod_lua
Posted by Stefan Fritsch <sf...@sfritsch.de>.
On Wednesday 29 December 2010, Andrew Farmer wrote:
> On 29 Dec 2010 at 01:25, Igor Galić wrote:
> > Please share your particularly ugly, involved, unaesthetic or
> > otherwise /wrong/ solutions done with mod_rewrite because it
> > was the only hammer available for the screws at that time ;)
>
> I came up with the following abomination a while back to disable
> hotlinking to files when the referrer doesn't match the request
> host (in
>
> conjunction with a setup that amounts to wildcard DNS):
> > RewriteEngine on
> > RewriteCond %{HTTP_REFERER} ://([^/]+)/
> > RewriteRule . - [E=RHOST:%1]
> >
> > RewriteCond %{REQUEST_URI} ^/(albums|cache)/
> > RewriteCond %{ENV:RHOST} !^$
> > RewriteCond %{HTTP_HOST}:%{ENV:RHOST} !^([^:]*):(\1)$
> > RewriteRule . - [F]
>
> The observant will notice a novel abuse of regex backreferences
> used here to implement string comparison. A reimplementation of
> the same configuration in Lua should be able to avoid this. :)
Not what Igor wanted, but this could be simplified a lot by using the
shiny new ability of RewriteCond to use ap_expr:
RewriteCond %{REQUEST_URI} ^/(albums|cache)/
RewriteCond expr "%{HTTP_REFERER} -strmatch '*://%{HTTP_HOST}/*'"
RewriteRule . - [F]
BTW, is it possible to implement rewrite maps with mod_lua? I think it
would be an interesting application of mod_lua if it could be used to
add rewrite map functions, and ap_expr functions and operators.
Re: Inspiration for mod_lua
Posted by Andrew Farmer <an...@hq.newdream.net>.
On 29 Dec 2010 at 01:25, Igor Galić wrote:
> Please share your particularly ugly, involved, unaesthetic or
> otherwise /wrong/ solutions done with mod_rewrite because it
> was the only hammer available for the screws at that time ;)
I came up with the following abomination a while back to disable
hotlinking to files when the referrer doesn't match the request host (in
conjunction with a setup that amounts to wildcard DNS):
> RewriteEngine on
> RewriteCond %{HTTP_REFERER} ://([^/]+)/
> RewriteRule . - [E=RHOST:%1]
>
> RewriteCond %{REQUEST_URI} ^/(albums|cache)/
> RewriteCond %{ENV:RHOST} !^$
> RewriteCond %{HTTP_HOST}:%{ENV:RHOST} !^([^:]*):(\1)$
> RewriteRule . - [F]
The observant will notice a novel abuse of regex backreferences used
here to implement string comparison. A reimplementation of the same
configuration in Lua should be able to avoid this. :)