You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Ian Holsman <li...@holsman.net> on 2006/04/17 08:13:06 UTC
It's that time of the year again
Google is about to start it's summer of code project
what does this mean for HTTP/APR ?
we need:
- mentors
and
- project ideas.
so.. if there is any niggly things or cool projects you haven't got
the time to do yourself, but could devote 2-3 hrs/week to help
someone else do, and could be accomplished by a good student in
10-12wks.
now's the time.
ideas so far (half joking):
- mod_ircd
- implementing a UDP protocol
- a caching module implement CML (cache-meta-language)
- a SEDA type MPM
last year's SoC produced:
- mod_smtpd
- mod_mbox enhancements
- mod_cache_requestor (which i don't think really took off)
and 2 active comitters.
so.. lets get brainstorming. Let's see HTTP get the prize for most
ideas (and beat those java weanies)
--Ian
--
Ian Holsman ++61-3-9877-0909
"in this place it takes all the running you can do, to keep in the
same place." - Lewis Caroll
Re: It's that time of the year again
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 4/16/06, Ian Holsman <li...@holsman.net> wrote:
> Google is about to start it's summer of code project
>
> what does this mean for HTTP/APR ?
>
> we need:
> - mentors
I'd be willing to help mentor.
> and
> - project ideas.
A few ideas:
in APR:
- Improve the build system so that it can generate win32 project
files automatically, instead of requiring us to maintain them by hand.
It also might be nice to allow generation of makefiles on win32, so
we can build via command line tools instead of requiring visual
studio.
- Add a logging API, abstracting the differences between syslog,
win32 event logs, and basic file backed logs. This project has the
potential to involve working with the Subversion project as well,
since it has a need for such an API.
in HTTPD:
- Extend the mod_proxy_fcgi code so that it can manage its own
worker processes, rather than requiring them to be managed externally.
Would most likely require a bit of refactoring inside of mod_proxy as
well.
-garrett
Re: It's that time of the year again
Posted by Esteban Pizzini <es...@gmail.com>.
On 4/17/06, Jeff McAdams <je...@iglou.com> wrote:
>
> Ian Holsman wrote:
> > ideas so far (half joking):
> > - mod_ircd
> > - implementing a UDP protocol
> > - a caching module implement CML (cache-meta-language)
> > - a SEDA type MPM
>
> mod_snmp would be very useful.
I have implemented snmp for Apache 2.0.x (mod-apache-snmp.sourceforge.net)
It't an implementation using net-snmp as agent.
I'm working on the module and there some features to add....
--
> Jeff McAdams
> "They that can give up essential liberty to obtain a
> little temporary safety deserve neither liberty nor safety."
> -- Benjamin Franklin
>
>
>
>
--
----------------------------------------------------------------------
Esteban Pizzini
(http://mod-apache-snmp.sourceforge.net)
Re: It's that time of the year again
Posted by Mads Toftum <ma...@toftum.dk>.
On Mon, Apr 17, 2006 at 02:25:46PM +0200, Jorge Schrauwen wrote:
> I've seen a mod_snmp somewhere:
> http://www.mod-snmp.com/mod_snmp.html
> don't know how if its free and for what version though.
>
SNMP module for Apache 1.3.x as you can see on www.mod-snmp.com
vh
Mads Toftum
--
`Darn it, who spiked my coffee with water?!' - lwall
Re: It's that time of the year again
Posted by Jorge Schrauwen <jo...@gmail.com>.
I've seen a mod_snmp somewhere:
http://www.mod-snmp.com/mod_snmp.html
don't know how if its free and for what version though.
On 4/17/06, Jeff McAdams <je...@iglou.com> wrote:
>
> Ian Holsman wrote:
> > ideas so far (half joking):
> > - mod_ircd
> > - implementing a UDP protocol
> > - a caching module implement CML (cache-meta-language)
> > - a SEDA type MPM
>
> mod_snmp would be very useful.
> --
> Jeff McAdams
> "They that can give up essential liberty to obtain a
> little temporary safety deserve neither liberty nor safety."
> -- Benjamin Franklin
>
>
>
>
--
~Jorge
Re: It's that time of the year again
Posted by Jeff McAdams <je...@iglou.com>.
Ian Holsman wrote:
> ideas so far (half joking):
> - mod_ircd
> - implementing a UDP protocol
> - a caching module implement CML (cache-meta-language)
> - a SEDA type MPM
mod_snmp would be very useful.
--
Jeff McAdams
"They that can give up essential liberty to obtain a
little temporary safety deserve neither liberty nor safety."
-- Benjamin Franklin
Re: It's that time of the year again
Posted by Davi Arnaut <da...@haxent.com.br>.
On Mon, 17 Apr 2006 16:13:06 +1000
Ian Holsman <li...@holsman.net> wrote:
> Google is about to start it's summer of code project
>
> what does this mean for HTTP/APR ?
>
> we need:
> - mentors
> and
> - project ideas.
>
> so.. if there is any niggly things or cool projects you haven't got
> the time to do yourself, but could devote 2-3 hrs/week to help
> someone else do, and could be accomplished by a good student in
> 10-12wks.
> now's the time.
>
>
> ideas so far (half joking):
> - mod_ircd
> - implementing a UDP protocol
> - a caching module implement CML (cache-meta-language)
> - a SEDA type MPM
>
As a CS student myself I was thinking about submitting a "mod_carp"
proposal. The general idea is to implement CARP (Cache Array Routing
Protocol [1]) as a mod_proxy_balancer (or mod_cache) method, habiliting
apache (mod_proxy) to be integrated into a cache hierarchy. Apache would
also be able to serve and parse a Proxy Array Membership Table.
Anyone interested ?
Davi Arnaut
[1]. http://icp.ircache.net/carp.txt
Re: It's that time of the year again
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Ian Holsman wrote:
>
> so.. lets get brainstorming. Let's see HTTP get the prize for most
> ideas (and beat those java weanies)
LOL! Ok, in all seriousness, error message localization is -sooo- long
overdue, and we have a perpetual problem with apr_status_t extention.
The next 2.0 release of apr really needs a 'register error results' for
a given set of codes... with a number-of-errors to be set aside, and the
appropriate callback to convert codes to error messages (notice the
intersection with the localization comment above.)
Re: It's that time of the year again
Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Mon, Apr 17, 2006 at 04:13:06PM +1000, Ian Holsman wrote:
> we need:
> - mentors
> and
> - project ideas.
Idea;
Add Apache Derby support in apr_dbd, either directly, via
c-jdbc or Sequoia (which is Apache licensed).
--
Colm MacCárthaigh Public Key: colm+pgp@stdlib.net
Re: It's that time of the year again
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 4/16/06, Ian Holsman <li...@holsman.net> wrote:
> Google is about to start it's summer of code project
>
> what does this mean for HTTP/APR ?
>
> we need:
> - mentors
I'd be willing to help mentor.
> and
> - project ideas.
A few ideas:
in APR:
- Improve the build system so that it can generate win32 project
files automatically, instead of requiring us to maintain them by hand.
It also might be nice to allow generation of makefiles on win32, so
we can build via command line tools instead of requiring visual
studio.
- Add a logging API, abstracting the differences between syslog,
win32 event logs, and basic file backed logs. This project has the
potential to involve working with the Subversion project as well,
since it has a need for such an API.
in HTTPD:
- Extend the mod_proxy_fcgi code so that it can manage its own
worker processes, rather than requiring them to be managed externally.
Would most likely require a bit of refactoring inside of mod_proxy as
well.
-garrett
Re: It's that time of the year again
Posted by Brian Akins <br...@turner.com>.
Garrett Rooney wrote:
> To really benchmark it's going
> to require more stuff than your average college student has lying
> around the house.
Simple dual opteron with GigE networking is more than sufficient. I can
"mentor" by testing some changes if somebody needs it.
--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies
Re: It's that time of the year again
Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Mon, Apr 17, 2006 at 10:15:09AM -0700, Garrett Rooney wrote:
> Perhaps, but AFAICT infra@ doesn't have this kind of thing lying
> around at the moment, so unless someone is going to step up with
> hardware people can use it's kind of a showstopper.
It's biased towards a single vendor, but SUN are opening up their QA
system to any open source projects interested. They have over 1000 boxes
of various different specs and some non-Sun OS's too. So that's a
potential option.
--
Colm MacCárthaigh Public Key: colm+pgp@stdlib.net
Re: It's that time of the year again
Posted by Mads Toftum <ma...@toftum.dk>.
On Mon, Apr 17, 2006 at 10:15:09AM -0700, Garrett Rooney wrote:
> Perhaps, but AFAICT infra@ doesn't have this kind of thing lying
> around at the moment, so unless someone is going to step up with
> hardware people can use it's kind of a showstopper.
>
Correct.
vh
Mads Toftum
--
`Darn it, who spiked my coffee with water?!' - lwall
Re: It's that time of the year again
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 4/17/06, Brian McCallister <br...@apache.org> wrote:
>
> On Apr 17, 2006, at 10:04 AM, Garrett Rooney wrote:
>
> > I suspect that a significant problem with this sort of project will be
> > lack of proper hardware for benchmarking purposes. From everything
> > I've heard it's not all that hard to totally saturate the kind of
> > networks you're likely to have sitting around your house with
> > commodity hardware and no real effort. To really benchmark it's going
> > to require more stuff than your average college student has lying
> > around the house.
>
> That would be one of the advantages of it being with the ASF -- in
> theory we can get access to some network gear which might be harder
> for a student to lay hands on.
Perhaps, but AFAICT infra@ doesn't have this kind of thing lying
around at the moment, so unless someone is going to step up with
hardware people can use it's kind of a showstopper.
-garrett
Re: It's that time of the year again
Posted by Brian McCallister <br...@apache.org>.
On Apr 17, 2006, at 10:04 AM, Garrett Rooney wrote:
> I suspect that a significant problem with this sort of project will be
> lack of proper hardware for benchmarking purposes. From everything
> I've heard it's not all that hard to totally saturate the kind of
> networks you're likely to have sitting around your house with
> commodity hardware and no real effort. To really benchmark it's going
> to require more stuff than your average college student has lying
> around the house.
That would be one of the advantages of it being with the ASF -- in
theory we can get access to some network gear which might be harder
for a student to lay hands on.
-Brian
Re: It's that time of the year again
Posted by Brian Akins <br...@turner.com>.
William A. Rowe, Jr. wrote:
> LOL - not to mention the converse. Even when eating all four 1GB network
> segments it's quite difficult to saturate anything but the NICs on one of
> Sun's T2000 loaner boxes until you add dynamic content to the mix :)
Try smaller files. like 1 byte. That is how I test a bunch of stuff
here. It's very hard to saturate a GigE when serving 1 byte files.
--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies
Re: It's that time of the year again
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Garrett Rooney wrote:
>
> I suspect that a significant problem with this sort of project will be
> lack of proper hardware for benchmarking purposes. From everything
> I've heard it's not all that hard to totally saturate the kind of
> networks you're likely to have sitting around your house with
> commodity hardware and no real effort. To really benchmark it's going
> to require more stuff than your average college student has lying
> around the house.
LOL - not to mention the converse. Even when eating all four 1GB network
segments it's quite difficult to saturate anything but the NICs on one of
Sun's T2000 loaner boxes until you add dynamic content to the mix :)
Re: It's that time of the year again
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 4/17/06, Colm MacCarthaigh <co...@stdlib.net> wrote:
> On Mon, Apr 17, 2006 at 12:34:29PM -0400, Rian A Hunter wrote:
> > I think a SoC project that profiles Apache (and finds out where we
> > fall short) so that we are able to compete with other lightweight HTTP
> > servers popping up these days would be a good endeavor for any CS
> > student.
>
> Right now, I'm getting 22k reqs/sec from Apache httpd, and 18k/sec from
> lighttpd. Simple things like using epoll, or the way the worker
> balancing is done have huge effects compared to the tiny improvements
> refactoring code can have.
>
> > This seems to be more viable for our threaded MPMs. For the prefork
> > MPM, maybe a goal for 10,000 connections might be impractical.
>
> With prefork I can generally push about 27,000 concurrent connections
> before things get hairy. With worker, I have a usable system up to
> 83,000 concurrent connections, without much effort.
>
> > I haven't done any benchmarks myself, I've just read results so anyone
> > correct me if I'm wrong.
>
> Dan Kegels page is years out of date, and was uninformed even when it
> wasn't :)
I suspect that a significant problem with this sort of project will be
lack of proper hardware for benchmarking purposes. From everything
I've heard it's not all that hard to totally saturate the kind of
networks you're likely to have sitting around your house with
commodity hardware and no real effort. To really benchmark it's going
to require more stuff than your average college student has lying
around the house.
-garrett
Re: It's that time of the year again
Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Mon, Apr 17, 2006 at 12:34:29PM -0400, Rian A Hunter wrote:
> I think a SoC project that profiles Apache (and finds out where we
> fall short) so that we are able to compete with other lightweight HTTP
> servers popping up these days would be a good endeavor for any CS
> student.
Right now, I'm getting 22k reqs/sec from Apache httpd, and 18k/sec from
lighttpd. Simple things like using epoll, or the way the worker
balancing is done have huge effects compared to the tiny improvements
refactoring code can have.
> This seems to be more viable for our threaded MPMs. For the prefork
> MPM, maybe a goal for 10,000 connections might be impractical.
With prefork I can generally push about 27,000 concurrent connections
before things get hairy. With worker, I have a usable system up to
83,000 concurrent connections, without much effort.
> I haven't done any benchmarks myself, I've just read results so anyone
> correct me if I'm wrong.
Dan Kegels page is years out of date, and was uninformed even when it
wasn't :)
--
Colm MacCárthaigh Public Key: colm+pgp@stdlib.net
Re: It's that time of the year again
Posted by Brian Akins <br...@turner.com>.
Rian A Hunter wrote:
> This seems to be more viable for our threaded MPMs. For the prefork
> MPM, maybe a
> goal for 10,000 connections might be impractical.
Using worker, we do "many thousands of connections" (ie, much more than
10k). I think Colm has published some numbers about his experiences.
--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies
Re: It's that time of the year again
Posted by Rian A Hunter <ri...@MIT.EDU>.
Quoting Ian Holsman <li...@holsman.net>:
> ideas so far (half joking):
> - mod_ircd
> - implementing a UDP protocol
> - a caching module implement CML (cache-meta-language)
> - a SEDA type MPM
http://www.kegel.com/c10k.html
I think a SoC project that profiles Apache (and finds out where we fall short)
so that we are able to compete with other lightweight HTTP servers popping up
these days would be a good endeavor for any CS student.
This seems to be more viable for our threaded MPMs. For the prefork
MPM, maybe a
goal for 10,000 connections might be impractical.
I haven't done any benchmarks myself, I've just read results so anyone correct
me if I'm wrong.
Rian
Re: It's that time of the year again
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Ian Holsman wrote:
>
> so.. lets get brainstorming. Let's see HTTP get the prize for most
> ideas (and beat those java weanies)
LOL! Ok, in all seriousness, error message localization is -sooo- long
overdue, and we have a perpetual problem with apr_status_t extention.
The next 2.0 release of apr really needs a 'register error results' for
a given set of codes... with a number-of-errors to be set aside, and the
appropriate callback to convert codes to error messages (notice the
intersection with the localization comment above.)
Re: It's that time of the year again
Posted by Joachim Zobel <jz...@heute-morgen.de>.
On Mon, 17 Apr 2006 16:13:06 +1000
Ian Holsman <li...@holsman.net> wrote:
> so.. if there is any niggly things or cool projects you haven't got
> the time to do yourself,
Yeah, my no 1. wish is a mod_tal (the existing sourceforge project was
abandomed before start) that:
1. acts as an output filter on xml data and
2. behaves streaming with respect to the xml data.
TAL is a spec for a template engine. For details on TAL see
http://www.zope.org/Wikis/DevSite/Projects/ZPT/TAL%20Specification%201.4
Just my 0.02 EUR.
Sincerely,
Joachim
Re: It's that time of the year again
Posted by Brian Akins <br...@turner.com>.
If anyone wants to be "mentored" in this (memcache and/or generalized
cache modules, new mod_http_cache) I am willing to be mentor.
Ian Holsman wrote:
> yeah.. thats the hard part of SoC.
> not coding it yourself in 20 minutes, and leaving it for your student to
> do ;-)
>>
>> Was just an idea, I can help in some way as well. I have a working
>> prototype now.
>>
>>
>> --Brian Akins
>> Lead Systems Engineer
>> CNN Internet Technologies
--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies
Re: It's that time of the year again
Posted by Ian Holsman <li...@holsman.net>.
yeah.. thats the hard part of SoC.
not coding it yourself in 20 minutes, and leaving it for your student
to do ;-)
On 18/04/2006, at 5:43 AM, Brian Akins wrote:
> Parin Shah wrote:
>
>> I have liked the idea of mod_memcached. I can work on it with you (if
>> we have Soc student for this project, I can work with him as well )
>
> To clarify, I really meant to say mod_memcache, the client part,
> not the server.
>
> Was just an idea, I can help in some way as well. I have a working
> prototype now.
>
>
> --
> Brian Akins
> Lead Systems Engineer
> CNN Internet Technologies
--
Ian Holsman ++61-3-9818-0132
Good judgment comes from experience, experience comes from bad judgment.
Re: It's that time of the year again
Posted by Brian Akins <br...@turner.com>.
Parin Shah wrote:
> I have liked the idea of mod_memcached. I can work on it with you (if
> we have Soc student for this project, I can work with him as well )
To clarify, I really meant to say mod_memcache, the client part, not the
server.
Was just an idea, I can help in some way as well. I have a working
prototype now.
--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies
Re: It's that time of the year again
Posted by Parin Shah <pa...@gmail.com>.
> An example I'd like to do (or mentor someone) is a mod_memcached that
> could serve as the basis of memcached based modules. It could handle
> all the configuration details, errors, and general heavy lifting of
> memcached. It would then be very easy to write other modules that had
> hooks into memcached (ie, a mod_cache provider).
>
I have liked the idea of mod_memcached. I can work on it with you (if
we have Soc student for this project, I can work with him as well )
Re: It's that time of the year again
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
FWIW, in our earlier discussions we discussed segmented keys, that any
provider wouldn't have to provide multiple keys (to be hooked) but would
have to support segmented keys such that (for example) three UUID's would
designate highest - narrower - narrowest lookup, and it would (of course)
be expected that a segment 0-only lookup would have clashes/multiple
entities.
What a program does with this situation is up to the user/consumer. Our
thought was that if (for example http caching) we first did a direct seek
on -one- page based on segment 0 & 1 (the host and the uri) we would get
back the variant headers. We could then perform a direct hit on segment
0 & 1 & 2 (the variation key) and get precisely the page they needed as
a two-hit-always operation. If the page doesn't vary, the first hit would
be the actual page. If variant also varies, well, guess we are SOL ;-)
Bill
Brian Akins wrote:
> An example I'd like to do (or mentor someone) is a mod_memcached that
> could serve as the basis of memcached based modules. It could handle
> all the configuration details, errors, and general heavy lifting of
> memcached. It would then be very easy to write other modules that had
> hooks into memcached (ie, a mod_cache provider).
>
> Perhaps to abstract it even further, perhaps mod_memcached should be a
> provider for a more generic caching framework. Not HTTP specific like
> mod_cache, but a general key based cache. Could be used for a variety
> of things (including an HTTP cache).
>
>
> typedef struct {
> apr_status_t (*cache_create)(ap_cache_t **instance, apr_table_t
> *params);
> apr_status_t (*cache_stats)(ap_cache_t *instance, ap_cache_stats_t
> **stats);
> apr_status_t (*cache_set)(ap_cache_t *instance, const char *key,
> void *val, apr_size_t len, apr_time_t expires);
> /*
> other stuff, like get, replace, etc.
> */
>
> } cache_provider_t;
>
> Just "thinking out loud."
>
>
> Also, mod_cache should be renamed mod_http_cache
>
Re: It's that time of the year again
Posted by Brian Akins <br...@turner.com>.
An example I'd like to do (or mentor someone) is a mod_memcached that
could serve as the basis of memcached based modules. It could handle
all the configuration details, errors, and general heavy lifting of
memcached. It would then be very easy to write other modules that had
hooks into memcached (ie, a mod_cache provider).
Perhaps to abstract it even further, perhaps mod_memcached should be a
provider for a more generic caching framework. Not HTTP specific like
mod_cache, but a general key based cache. Could be used for a variety
of things (including an HTTP cache).
typedef struct {
apr_status_t (*cache_create)(ap_cache_t **instance, apr_table_t
*params);
apr_status_t (*cache_stats)(ap_cache_t *instance, ap_cache_stats_t
**stats);
apr_status_t (*cache_set)(ap_cache_t *instance, const char *key,
void *val, apr_size_t len, apr_time_t expires);
/*
other stuff, like get, replace, etc.
*/
} cache_provider_t;
Just "thinking out loud."
Also, mod_cache should be renamed mod_http_cache
--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies
Re: It's that time of the year again
Posted by Graham Dumpleton <gr...@dscpl.com.au>.
On 17/04/2006, at 6:43 PM, Nick Kew wrote:
> On Monday 17 April 2006 07:45, Graham Dumpleton wrote:
>> On 17/04/2006, at 4:27 PM, Nick Kew wrote:
>>> On Monday 17 April 2006 07:13, Ian Holsman wrote:
>>>
>>> OK, let's play ...
>>>
>>> - Language bindings (mod_[perl|python|etc]) for new goodies
>>> like DBD and XMLNS
>>
>> This possibly ties up with something which I was intending to one day
>> implement
>> in mod_python. That is to export from mod_python, as optional
>> functions, functions
>> which would allow one to get access to the Python interpreter
>> instances plus other
>> functions for accessing/creating Python wrappers for request, server
>> and filter
>> objects.
>>
>> The original intent of doing this was so that module writers using C
>> code could
>> access the Python interpreters already managed by mod_python and then
>> provide a means in their own module of being able to use Python code.
>
> We actually have an analagous setup working with Tcl. It's quite a
> small
> module, but works well to run Tcl script embedded in HTML pages, and
> help the Client upgrade from the vignette system they previously used.
> A separate module - used by the interpreter - provides Tcl bindings
> for DBD.
>
> My thought on doing the same with Python is that it really
> shouldn't need
> the full baggage of mod_python just to do this. All they really
> have in
> common is the python interpretors. So maybe the architecture for this
> could look something like:
>
> mod_python_base (manage the python interpreters)
> mod_python (as now, less what's moved to python_base) - big
> mod_python_embedded (python interpreters for C programmers) - small
There isn't actually much in mod_python that could be extracted out
into a
base or embedded package. Pretty well all the C code would still be
required in order to support the basic stuff that would be exportable as
optional functions. The only bits that might not move are code for
filter and
handler directives for registering and then invoking handlers as part of
normal Apache request processing phases. This would be at most a few
hundred lines of code.
In terms of the package as a whole and what possibly shouldn't be in
mod_python are the mod_python.publisher and mod_python.psp handlers
which are in effect a layer over the core mod_python handler mechanisms.
These amount to mostly Python code, although mod_python.psp has a
C code component, but that isn't part of the Apache module, but a
loadable
Python extension module. Thus neither actually affected the footprint of
the Apache module when not in use.
Thus, there probably is very little to be gained from doing a split
and in
practice it would probably make it harder to manage the code base as
far as releases and maintenance.
>> The particular use case I was seeing this as being targeted at was so
>> as to be
>> able to implement a mod_dav_python. That is, mod_dav_python is
>> primarily
>> a C based module which hooks into mod_dav and bridges the C hooks
>> into hooks
>> implemented in Python. The mod_dav_python would need to provide
>> Python
>> wrapper classes for all the mod_dav structures, but at least it
>> doesn't have to
>> duplicate what mod_python does in the way of interpreter
>> management and
>> wrappers for request object etc. Overall this could allow a mod_dav
>> filesystem
>> to be implemented in Python.
>
> Hmmm. I'm not sure I see what you mean. Providing python hooks in
> mod_dav
> ops would surely be a relatively simple extension to the existing
> mod_python?
The issue is more that mod_dav support shouldn't belong in mod_python
itself. It should be ignorant of other Apache modules. If mod_python
itself
had dependencies on other modules, it makes it harder to develop as you
have to bring into the core development team people who have intimate
knowledge of the other packages. The only exception we have made to that
ideal is that the request object in mod_python in unreleased code
hooks in
optional functions of mod_ssl because of its basic importance. That was
purely out of convenience as it had been demonstrated how to make a
standalone Python module which achieved the same end.
> XMLNS bindings would enable people to script SAX2 callback events
> in Python,
> and mix-and-match with C modules, all running in an XMLNS filter.
> How does
> that look from your PoV?
Can't see why it couldn't be done. The next version of mod_python hooks
into mod_includes to allow Python code in SSI files. What you are
talking
about doesn't sound much different.
>> The same set of optional functions exported from mod_python may be
>> useful for
>> implementing your suggestion. It is certainly a preferable approach
>> to making
>> mod_python understand something like mod_dav and embedding the
>> support
>> within it as standard.
>
> Right, yes. It looks like a potential fit:-)
Only problem is that there is a bit of cleanup work in internals of
mod_python
before the optional functions could be put in place. I'm slowly
getting there
though. :-)
Anyway, in terms of the request for projects, if these optional
functions were
exposed in mod_python, am sure it could be a trigger for a number of
interesting projects.
Graham
Re: It's that time of the year again
Posted by Nick Kew <ni...@webthing.com>.
On Monday 17 April 2006 07:45, Graham Dumpleton wrote:
> On 17/04/2006, at 4:27 PM, Nick Kew wrote:
> > On Monday 17 April 2006 07:13, Ian Holsman wrote:
> >
> > OK, let's play ...
> >
> > - Language bindings (mod_[perl|python|etc]) for new goodies
> > like DBD and XMLNS
>
> This possibly ties up with something which I was intending to one day
> implement
> in mod_python. That is to export from mod_python, as optional
> functions, functions
> which would allow one to get access to the Python interpreter
> instances plus other
> functions for accessing/creating Python wrappers for request, server
> and filter
> objects.
>
> The original intent of doing this was so that module writers using C
> code could
> access the Python interpreters already managed by mod_python and then
> provide a means in their own module of being able to use Python code.
We actually have an analagous setup working with Tcl. It's quite a small
module, but works well to run Tcl script embedded in HTML pages, and
help the Client upgrade from the vignette system they previously used.
A separate module - used by the interpreter - provides Tcl bindings for DBD.
My thought on doing the same with Python is that it really shouldn't need
the full baggage of mod_python just to do this. All they really have in
common is the python interpretors. So maybe the architecture for this
could look something like:
mod_python_base (manage the python interpreters)
mod_python (as now, less what's moved to python_base) - big
mod_python_embedded (python interpreters for C programmers) - small
>
> The particular use case I was seeing this as being targeted at was so
> as to be
> able to implement a mod_dav_python. That is, mod_dav_python is primarily
> a C based module which hooks into mod_dav and bridges the C hooks
> into hooks
> implemented in Python. The mod_dav_python would need to provide Python
> wrapper classes for all the mod_dav structures, but at least it
> doesn't have to
> duplicate what mod_python does in the way of interpreter management and
> wrappers for request object etc. Overall this could allow a mod_dav
> filesystem
> to be implemented in Python.
Hmmm. I'm not sure I see what you mean. Providing python hooks in mod_dav
ops would surely be a relatively simple extension to the existing mod_python?
Though it does sound like a pretty similar task to providing DBD bindings.
Maybe this is because I've not looked at mod_python from a C perspective.
XMLNS bindings would enable people to script SAX2 callback events in Python,
and mix-and-match with C modules, all running in an XMLNS filter. How does
that look from your PoV?
> The same set of optional functions exported from mod_python may be
> useful for
> implementing your suggestion. It is certainly a preferable approach
> to making
> mod_python understand something like mod_dav and embedding the support
> within it as standard.
Right, yes. It looks like a potential fit:-)
--
Nick Kew
Re: It's that time of the year again
Posted by Graham Dumpleton <gr...@dscpl.com.au>.
On 17/04/2006, at 4:27 PM, Nick Kew wrote:
> On Monday 17 April 2006 07:13, Ian Holsman wrote:
>
> OK, let's play ...
>
> - Language bindings (mod_[perl|python|etc]) for new goodies
> like DBD and XMLNS
This possibly ties up with something which I was intending to one day
implement
in mod_python. That is to export from mod_python, as optional
functions, functions
which would allow one to get access to the Python interpreter
instances plus other
functions for accessing/creating Python wrappers for request, server
and filter
objects.
The original intent of doing this was so that module writers using C
code could
access the Python interpreters already managed by mod_python and then
provide a means in their own module of being able to use Python code.
The particular use case I was seeing this as being targeted at was so
as to be
able to implement a mod_dav_python. That is, mod_dav_python is primarily
a C based module which hooks into mod_dav and bridges the C hooks
into hooks
implemented in Python. The mod_dav_python would need to provide Python
wrapper classes for all the mod_dav structures, but at least it
doesn't have to
duplicate what mod_python does in the way of interpreter management and
wrappers for request object etc. Overall this could allow a mod_dav
filesystem
to be implemented in Python.
The same set of optional functions exported from mod_python may be
useful for
implementing your suggestion. It is certainly a preferable approach
to making
mod_python understand something like mod_dav and embedding the support
within it as standard.
Graham
Re: It's that time of the year again
Posted by Mads Toftum <ma...@toftum.dk>.
On Mon, Apr 17, 2006 at 07:27:08AM +0100, Nick Kew wrote:
> - Update apxs to search web, download & verify modules,
> get security and license info.
>
More details from the discussions at apachecon EU last year:
http://mail-archives.apache.org/mod_mbox/httpd-dev/200507.mbox/%3c4DDC291C-C99C-424C-8149-E9F2A027F721@temme.net%3e
vh
Mads Toftum
--
`Darn it, who spiked my coffee with water?!' - lwall
Re: It's that time of the year again
Posted by Graham Dumpleton <gr...@dscpl.com.au>.
On 17/04/2006, at 4:27 PM, Nick Kew wrote:
> On Monday 17 April 2006 07:13, Ian Holsman wrote:
>
> OK, let's play ...
>
> - Language bindings (mod_[perl|python|etc]) for new goodies
> like DBD and XMLNS
This possibly ties up with something which I was intending to one day
implement
in mod_python. That is to export from mod_python, as optional
functions, functions
which would allow one to get access to the Python interpreter
instances plus other
functions for accessing/creating Python wrappers for request, server
and filter
objects.
The original intent of doing this was so that module writers using C
code could
access the Python interpreters already managed by mod_python and then
provide a means in their own module of being able to use Python code.
The particular use case I was seeing this as being targeted at was so
as to be
able to implement a mod_dav_python. That is, mod_dav_python is primarily
a C based module which hooks into mod_dav and bridges the C hooks
into hooks
implemented in Python. The mod_dav_python would need to provide Python
wrapper classes for all the mod_dav structures, but at least it
doesn't have to
duplicate what mod_python does in the way of interpreter management and
wrappers for request object etc. Overall this could allow a mod_dav
filesystem
to be implemented in Python.
The same set of optional functions exported from mod_python may be
useful for
implementing your suggestion. It is certainly a preferable approach
to making
mod_python understand something like mod_dav and embedding the support
within it as standard.
Graham
Re: It's that time of the year again
Posted by Nick Kew <ni...@webthing.com>.
On Monday 17 April 2006 07:13, Ian Holsman wrote:
> ideas so far (half joking):
> - mod_ircd
> - implementing a UDP protocol
> - a caching module implement CML (cache-meta-language)
> - a SEDA type MPM
OK, let's play ...
- Language bindings (mod_[perl|python|etc]) for new goodies
like DBD and XMLNS
- APR build modularisation and dlloading
- Update apxs to search web, download & verify modules,
get security and license info.
--
Nick Kew
Re: It's that time of the year again
Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Mon, Apr 17, 2006 at 06:44:11PM +1000, John Vandenberg wrote:
> A "cool" project that appears to needs a coder is mod_bittorrent.
There's already a mod_bittorrent, but it only produces .torrent files
dynamically, it doesn't act as as a seed or participate in the p2p.
There's mod_torrent too which has ceased development.
A working bittorrent module for httpd is directly useful to me, and it's
been on my personal TODO for a while now, so it's something I'd be happy
to mentor.
> The initial objective is to dynamically build a torrent file for a
> user requested file. This could be integrated into httpd by sending
> large files as torrents when the user agent states that it accepts
> application/x-bittorrent, and/or when the server notices that the
> number of downloads for the file has risen rapidly.
Nah, that's boring and far too trivial imo. We need a bittorrent
protocol implementation, the host needs to act as a seed too :)
--
Colm MacCárthaigh Public Key: colm+pgp@stdlib.net
Re: It's that time of the year again
Posted by John Vandenberg <ja...@gmail.com>.
On 4/17/06, Ian Holsman <li...@holsman.net> wrote:
> Google is about to start it's summer of code project
>
> what does this mean for HTTP/APR ?
>
> we need:
> - mentors
> and
> - project ideas.
>
> so.. if there is any niggly things or cool projects you haven't got
> the time to do yourself, but could devote 2-3 hrs/week to help
> someone else do, and could be accomplished by a good student in
> 10-12wks.
> now's the time.
>
>
> ideas so far (half joking):
> - mod_ircd
> - implementing a UDP protocol
> - a caching module implement CML (cache-meta-language)
> - a SEDA type MPM
A "cool" project that appears to needs a coder is mod_bittorrent. The
initial objective is to dynamically build a torrent file for a user
requested file. This could be integrated into httpd by sending large
files as torrents when the user agent states that it accepts
application/x-bittorrent, and/or when the server notices that the
number of downloads for the file has risen rapidly.
This would complement the incomplete firepuddle project from last years SoC.
http://mod-torrent.sourceforge.net/
--
John
Re: It's that time of the year again
Posted by Parin Shah <pa...@gmail.com>.
> - mod_cache_requestor (which i don't think really took off)
> and 2 active comitters.
I still haven't given up on it. :-) I am trying to remove the libcurl
dependency by creating mocked up connection and request. hopefully, it
would take off one day :-)
Re: It's that time of the year again
Posted by Nick Kew <ni...@webthing.com>.
On Monday 17 April 2006 07:13, Ian Holsman wrote:
> ideas so far (half joking):
> - mod_ircd
> - implementing a UDP protocol
> - a caching module implement CML (cache-meta-language)
> - a SEDA type MPM
OK, let's play ...
- Language bindings (mod_[perl|python|etc]) for new goodies
like DBD and XMLNS
- APR build modularisation and dlloading
- Update apxs to search web, download & verify modules,
get security and license info.
--
Nick Kew