You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by "Philippe M . Chiasson" <go...@cpan.org> on 2001/02/07 01:03:06 UTC
[mod_proxy] Help offered
Hi, I've been following the current mod_proxy thread and there are a few
contribution I'd like to make.
First of all, I am an avid follower of new-httpd but I think this is my
first post, so my apologies if I step on anybodys toes.
I think the whole discussion makes a lot of sense. There are some modules
in the core httpd distro (mod_proxy, mod_rewrite, mod_negociation, etc) that
are pretty big and almost have their own development cycle. And if it
slows down or even stops the release of a new httpd release, I think it's
pretty clear they should belong in their own repositories.
My interest in the mod_proxy discussion is because I had extensive experience
with it's internals as I had to alter it to proxy for a proprietary data
storage/retrieval system we are stuck with.
I know of at least 2 very different uses for the mod_proxy module currently.
The first one is to act as a caching HTTP proxy, but the second is to make a
lightweight doc server proxy some requests to a heavyweight server (mod_perl).
There 2 jobs share some common code/functionnality but there are also quite
different.
I think it would be wise to consider splitting these 2 tasks up to.
Anyways, am I officially offering my services to help out write some code. I already
started writing a simple lightweight proxy module for the light/heavy server setup for
1.3.x a while ago.
I guess I just don't want to end up with apache without a working mod_proxy that I really
need and use all the time. I understand the past problem with 'official' maintainers coming
in and out, so I will not make the mistake of offering to maintain it.
Just figure out how it should be handled and I will happilly help out however I can. I've
been holding out for too long.
P.S. Thanks for making a kickass httpd ;-)
--
+-----------------------------------------------+
| Philippe M. Chiasson <go...@cpan.org> |
| SmartWorker http://www.smartworker.org |
| IM : gozerhbe ICQ : gozer/18279998 |
| 64.8% computer corrupt according to |
| http://www.freespeech.org/ljk/purity.html |
+-----------------------------------------------+
Don't know how to handle magic of type '%s' : The internal
handling of magical variables has been cursed.
-- perldiag(1)
perl -e '$$=\${gozer};{$_=unpack(P26,pack(L,$$));/^Just Another Perl Hacker!\n$/&&print||$$++&&redo}'
Re: [mod_proxy] Help offered
Posted by Graham Leggett <mi...@sharp.fm>.
"Theo E. Schlossnagle" wrote:
> What you say is deceiving. Squid is much more effecient for high volume
> proxying than Apache 1.3.x.
Very likely - but the point behind putting a mod_proxy Apache in front
of a website is to make a complex website manageable, it had nothing to
do with performance.
The killer app for us was to allow Windows based ASP sites to coexist in
the same URL space as Netscape Enterprise server sites, and the same URL
space as static content. It allowed us to get rid of the awful and
enormously irritating prefixes in front of many of the website URLs, and
allowed us to define a workable URL strategy that was not tied in any
way to webserver architecture.
> Most big sites that _really_ need something like this, just make these changes
> to suit their needs.
In our case we were a big site, and we did make changes to suit our
needs - however Squid had nothing close to the features we needed. We
needed a webserver to front our websites, not a proxy.
> Squid's purpose in life it to be transparent. So, you do loose some features
> like being able to use mod_rewrite to "glue" several backend machines together
> into a unified namespace on the front. Hacking mod_rewrite functionality into
> Squid is a much harder problem IMHO.
Exactly - which is why we didn't use it.
Regards,
Graham
--
Re: [mod_proxy] Help offered
Posted by Greg Stein <gs...@lyra.org>.
On Thu, Feb 15, 2001 at 02:05:41PM -0500, Theo E. Schlossnagle wrote:
>...
> It makes complete sense to me to partition mod_proxy from the httpd-2.0 tree.
> Once it is lean and mean and performs _only_ the things a core module should,
> then stick it back in. If people _need_ that functionality, then for now they
> download it separately.
This was done several nights ago. Apache 2.0 no longer includes mod_proxy,
nor will it be held up by proxy development. If the proxy gets beefed up by
the time 2.0 goes to release, then cool. Otherwise, we'll release 2.0
without the proxy and people can separately fetch the alpha/beta release of
the proxy (wherever it is, in its dev cycle).
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: [mod_proxy] Help offered
Posted by Greg Stein <gs...@lyra.org>.
On Thu, Feb 15, 2001 at 02:05:41PM -0500, Theo E. Schlossnagle wrote:
>...
> It makes complete sense to me to partition mod_proxy from the httpd-2.0 tree.
> Once it is lean and mean and performs _only_ the things a core module should,
> then stick it back in. If people _need_ that functionality, then for now they
> download it separately.
This was done several nights ago. Apache 2.0 no longer includes mod_proxy,
nor will it be held up by proxy development. If the proxy gets beefed up by
the time 2.0 goes to release, then cool. Otherwise, we'll release 2.0
without the proxy and people can separately fetch the alpha/beta release of
the proxy (wherever it is, in its dev cycle).
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: [mod_proxy] Help offered
Posted by "Theo E. Schlossnagle" <th...@omniti.com>.
Since we are talking about mod_proxy, I will assume that you are referring to
the 1.3.x mod_proxy -- as it doesn't work with 2.0 (yet?).
What you say is deceiving. Squid is much more effecient for high volume
proxying than Apache 1.3.x. It's implementation supports several thousands of
concurrently open sessions. Also, it is quite trvial to modify Squid to
handle "virtual hosting". The point is that squid is transparent... It
doesn't care what is behind it. With relatively small modifications, you can
put Squid in front of several URL variants and it will work with logging to
seperate files. It is also trivial to hack Squid to support Apache style
commonlog formatting strings.
Most big sites that _really_ need something like this, just make these changes
to suit their needs.
Squid's purpose in life it to be transparent. So, you do loose some features
like being able to use mod_rewrite to "glue" several backend machines together
into a unified namespace on the front. Hacking mod_rewrite functionality into
Squid is a much harder problem IMHO.
It makes complete sense to me to partition mod_proxy from the httpd-2.0 tree.
Once it is lean and mean and performs _only_ the things a core module should,
then stick it back in. If people _need_ that functionality, then for now they
download it separately.
I am starting to see mod_proxy hold up httpd-2.0 development -- that scares
me.
Graham Leggett wrote:
> Squid and NSPS are only useful in trivial reverse proxy situations,
> where you have one frontend website URL, and your entire webspace is
> proxied to one or more HTTP backends. Try and do anything more
> complicated and you're stuck.
--
Theo Schlossnagle
1024D/A8EBCF8F/13BD 8C08 6BE2 629A 527E 2DC2 72C2 AD05 A8EB CF8F
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7
Re: [mod_proxy] Help offered
Posted by Graham Leggett <mi...@sharp.fm>.
Greg Stein wrote:
> My point is that people have alternatives. I was under the impression that
> Squid didn't have reverse proxy support, which meant Apache was the only
> "real" one out there (thus making our rev proxy support relatively important
> in the scheme of things). Finding that Squid (whose sole purpose in life is
> proxying) has reverse proxy changes my point of view dramatically.
When we were investigating reverse proxies a while back for Ericsson, we
looked at both Squid and at Netscape Proxy server as solutions to our
needs. Both of them sucked bigtime.
Both Squid and NSPS are forward proxies with a reverse proxy feature.
Apache is a webserver with a reverse proxy feature. In any multi-tier
architecture the best solution is one that behaves like a webserver.
Apache won in our feasibility study because it could handle multiple and
separate virtual hosts, and could log hits separately. It could also
combine static content, dynamic CGI content, and backend reverse proxied
sites into the same webspace. Neither Squid nor NSPS could do any of
this - which was a showstopper for us.
Squid and NSPS are only useful in trivial reverse proxy situations,
where you have one frontend website URL, and your entire webspace is
proxied to one or more HTTP backends. Try and do anything more
complicated and you're stuck.
Regards,
Graham
--
Re: [mod_proxy] Help offered
Posted by Bill Stoddard <bi...@wstoddard.com>.
> > > Feh. If that's so, then there is even more reason to move the proxy out
of
> > > the core and into a separate CVS repository for independent development.
> >
> > I disagree. Squid is a heavy weight 'solution' for folks just interested
in proxy pass (aka
> > "reverse" proxy).
>
> And Apache is light weight? :-)
>
<snip>
>
> While it would be nice for us to have it (due to licensing, features,
> whatever), I don't see it as critical or core to our main goal of being a
> web server.
I would be happy if we just had a good efficient proxy pass module in core
Apache. The rest can go into a seperate repository.
Bill
Re: [mod_proxy] Help offered
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Feb 06, 2001 at 10:11:51PM -0500, Bill Stoddard wrote:
> > On Tue, Feb 06, 2001 at 04:13:31PM -0800, Sander van Zoest wrote:
> > >...
> > > Although it should considered, I think the power of an apache proxy server
> > > would be the fact that it could do both of these tasks. I would consider
> > > a proxy type of project to be more along the lines of what Dean Gaudet has
> > > been refering to as an "http router". Everyone is doing this one way or
> > > another and this is also part of squid now (with their accelerated support).
> >
> > Woah! Squid now has reverse proxy support?
> >
> > Feh. If that's so, then there is even more reason to move the proxy out of
> > the core and into a separate CVS repository for independent development.
>
> I disagree. Squid is a heavy weight 'solution' for folks just interested in proxy pass (aka
> "reverse" proxy).
And Apache is light weight? :-)
Are you arguing to keep mod_proxy in the core? If so, then I'm also assuming
that you're signing up to assist in maintaining it. :-)
My point is that people have alternatives. I was under the impression that
Squid didn't have reverse proxy support, which meant Apache was the only
"real" one out there (thus making our rev proxy support relatively important
in the scheme of things). Finding that Squid (whose sole purpose in life is
proxying) has reverse proxy changes my point of view dramatically.
While it would be nice for us to have it (due to licensing, features,
whatever), I don't see it as critical or core to our main goal of being a
web server. Moving it to its own CVS module with its own dev staff can only
help the thing. Keep it in the core (in its current, unmaintained state) has
all kinds of problems.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: [mod_proxy] Help offered
Posted by Bill Stoddard <bi...@wstoddard.com>.
> On Tue, Feb 06, 2001 at 04:13:31PM -0800, Sander van Zoest wrote:
> >...
> > Although it should considered, I think the power of an apache proxy server
> > would be the fact that it could do both of these tasks. I would consider
> > a proxy type of project to be more along the lines of what Dean Gaudet has
> > been refering to as an "http router". Everyone is doing this one way or
> > another and this is also part of squid now (with their accelerated support).
>
> Woah! Squid now has reverse proxy support?
>
> Feh. If that's so, then there is even more reason to move the proxy out of
> the core and into a separate CVS repository for independent development.
I disagree. Squid is a heavy weight 'solution' for folks just interested in proxy pass (aka
"reverse" proxy).
Bill
Re: [mod_proxy] Help offered
Posted by "Philippe M . Chiasson" <go...@cpan.org>.
On Tue, Feb 06, 2001 at 06:25:25PM -0800, Greg Stein wrote:
> On Tue, Feb 06, 2001 at 04:13:31PM -0800, Sander van Zoest wrote:
> >...
> > Although it should considered, I think the power of an apache proxy server
> > would be the fact that it could do both of these tasks. I would consider
> > a proxy type of project to be more along the lines of what Dean Gaudet has
> > been refering to as an "http router". Everyone is doing this one way or
> > another and this is also part of squid now (with their accelerated support).
>
> Woah! Squid now has reverse proxy support?
>
> Feh. If that's so, then there is even more reason to move the proxy out of
> the core and into a separate CVS repository for independent development. If
> something is unmaintained *and* duplicates the functionality of other, free
> software projects out there, then why should it be in the core? I'm fine
> with competing open source projects, but we aren't competing with our proxy:
> we're just letting a bunch of code be subject to horrible bit rot and
> misleading people who think we have a proxy.
>
> > If some people add a reliable cache that talks HTCP and ICP and whatever they
> > feel is useful that would be a bonus. Having access via different languages to
> > the proxy/router stream can be very beneficial as well.
>
> All of that stuff is worthwhile, and would make great sense to develop
> within a separate CVS module with different sets of committers and
> interested parties. Tying the proxy releases in with the Apache core release
> will simply ensure that rapid development of a full-on proxy is going to be
> very painful.
100% agreed. And with the new hook API, it means mod_proxy could be written in a very
modular way, making it trivial to extend it in many different ways. Not something possible
with the current mod_proxy implementation.
And even if Squid is nice and feature rich, sometimes I just want an extra module in my
already running httpd, than have to deal with yet another server to manage.
Just my 0.02 $
> Cheers,
> -g
>
> --
> Greg Stein, http://www.lyra.org/
>
>
--
+-----------------------------------------------+
| Philippe M. Chiasson <go...@cpan.org> |
| SmartWorker http://www.smartworker.org |
| IM : gozerhbe ICQ : gozer/18279998 |
| 64.8% computer corrupt according to |
| http://www.freespeech.org/ljk/purity.html |
+-----------------------------------------------+
Men are only as good as their technical development allows
them to be.
-- George Orwell
perl -e '$$=\${gozer};{$_=unpack(P26,pack(L,$$));/^Just Another Perl Hacker!\n$/&&print||$$++&&redo}'
Re: [mod_proxy] Help offered
Posted by Greg Stein <gs...@lyra.org>.
On Wed, Feb 07, 2001 at 07:22:44AM +0000, Tony Finch wrote:
> Greg Stein <gs...@lyra.org> wrote:
> >
> >Woah! Squid now has reverse proxy support?
>
> What do you mean "now"? It has had it for years. Demon's Homepages
Fine. I wasn't aware of it.
>...
> >Feh. If that's so, then there is even more reason to move the proxy out of
> >the core and into a separate CVS repository for independent development. If
> >something is unmaintained *and* duplicates the functionality of other, free
> >software projects out there, then why should it be in the core?
>
> Squid is GPLed. Squid doesn't have SSL (afaik -- this may have changed
> recently). Squid's authentication support is comparatively limited.
All great reasons for an Apache one. How about volunteering? :-)
Just because there are failings in Squid doesn't mean ours is suddenly
better. Somebody has to put some elbow grease into it, and do so
consistently. Despite numerous attempts at this, nobody has yet done so. We
go to throw it out, and somebody appears for a week. Six months go by. We go
to toss it again, and somebody appears for two weeks this time.
We're not going to get a proxy that is competitive to Squid by continuing on
the current road. Our current arrangement is not ideal for proxy support, so
let's change it.
Move mod_proxy to its own CVS module. If somebody is actually going to spend
time working on it, then it certainly doesn't matter what CVS module it
happens to live within. The only difference is when we distribute our final
version of Apache. If mod_proxy is ready, then it can be bundled. If it
isn't ready, then it isn't bundled. Simple as that. Keeping it in will not
magically make it ready. Moving it out won't either. But moving it will
reduce the problems, fixing efforts, and the deception that Apache has proxy
support. Moving it out could also open the codebase to more developers (I am
suspicious of this because those developers aren't sending patches, so I
don't see that they'd be signing up as committers either).
Where are the negatives to moving it to its own CVS module?
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: [mod_proxy] Help offered
Posted by Tony Finch <do...@dotat.at>.
Greg Stein <gs...@lyra.org> wrote:
>
>Woah! Squid now has reverse proxy support?
What do you mean "now"? It has had it for years. Demon's Homepages
system (designed in 1996) uses Squid reverse proxies. You also need
reverse proxy functionality to be a transparent proxy, which Squid can
also do.
>Feh. If that's so, then there is even more reason to move the proxy out of
>the core and into a separate CVS repository for independent development. If
>something is unmaintained *and* duplicates the functionality of other, free
>software projects out there, then why should it be in the core?
Squid is GPLed. Squid doesn't have SSL (afaik -- this may have changed
recently). Squid's authentication support is comparatively limited.
Tony.
--
f.a.n.finch fanf@covalent.net dot@dotat.at
"Dead! And yet there he stands!"
Re: [mod_proxy] Help offered
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Feb 06, 2001 at 04:13:31PM -0800, Sander van Zoest wrote:
>...
> Although it should considered, I think the power of an apache proxy server
> would be the fact that it could do both of these tasks. I would consider
> a proxy type of project to be more along the lines of what Dean Gaudet has
> been refering to as an "http router". Everyone is doing this one way or
> another and this is also part of squid now (with their accelerated support).
Woah! Squid now has reverse proxy support?
Feh. If that's so, then there is even more reason to move the proxy out of
the core and into a separate CVS repository for independent development. If
something is unmaintained *and* duplicates the functionality of other, free
software projects out there, then why should it be in the core? I'm fine
with competing open source projects, but we aren't competing with our proxy:
we're just letting a bunch of code be subject to horrible bit rot and
misleading people who think we have a proxy.
> If some people add a reliable cache that talks HTCP and ICP and whatever they
> feel is useful that would be a bonus. Having access via different languages to
> the proxy/router stream can be very beneficial as well.
All of that stuff is worthwhile, and would make great sense to develop
within a separate CVS module with different sets of committers and
interested parties. Tying the proxy releases in with the Apache core release
will simply ensure that rapid development of a full-on proxy is going to be
very painful.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: [mod_proxy] Help offered
Posted by "Theo E. Schlossnagle" <je...@omniti.com>.
James Sutherland wrote:
> Plan:
>
> Various policies available to decide how to handle each request:
> * Backend server load
> * Source IP - redirect to a nearer mirror, perhaps
> * URL requested - separate static and CGI requests
> * Cache - if we've got a suitable cached item, send that
>
> Thoughts??
Sure, mod_backhand. http://www.backhand.org/mod_backhand/
It should do exactly what you want... Once it is ported to 2.0 that is :-) It
is actively maintained, and though provides a much simpler proxy pass (read:
more limited), it does process at least some 1.1 (e.g. chunked encoding). And
as reverse proxying goes it _much_much_ smarter than mod_proxy. It has
connection pooling and transparent keep alive upgrades on the back-side. It
can proxy hundreds of requests from different clients over the same HTTP
session to a back-end server.
--
Theo Schlossnagle
1024D/A8EBCF8F/13BD 8C08 6BE2 629A 527E 2DC2 72C2 AD05 A8EB CF8F
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7
Re: [mod_proxy] Help offered
Posted by "Theo E. Schlossnagle" <je...@omniti.com>.
James Sutherland wrote:
> Plan:
>
> Various policies available to decide how to handle each request:
> * Backend server load
> * Source IP - redirect to a nearer mirror, perhaps
> * URL requested - separate static and CGI requests
> * Cache - if we've got a suitable cached item, send that
>
> Thoughts??
Sure, mod_backhand. http://www.backhand.org/mod_backhand/
It should do exactly what you want... Once it is ported to 2.0 that is :-) It
is actively maintained, and though provides a much simpler proxy pass (read:
more limited), it does process at least some 1.1 (e.g. chunked encoding). And
as reverse proxying goes it _much_much_ smarter than mod_proxy. It has
connection pooling and transparent keep alive upgrades on the back-side. It
can proxy hundreds of requests from different clients over the same HTTP
session to a back-end server.
--
Theo Schlossnagle
1024D/A8EBCF8F/13BD 8C08 6BE2 629A 527E 2DC2 72C2 AD05 A8EB CF8F
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7
Re: [mod_proxy] Help offered
Posted by James Sutherland <ja...@cam.ac.uk>.
On Wed, 7 Feb 2001, Gabriel Russell wrote:
> At 01:11 PM 2/7/2001 +0000, James Sutherland wrote:
Eh? The next quote was not mine!
> > > > There 2 jobs share some common code/functionnality but there are also
> > quite
> > > > different.
>
> Where they do share common code functionality, it's often not totally
> perfect. Looking at the code I see no compelling reason to have one code
> base for these two functions. Caching is the one obvious section that is
> clearly sharable by the two sections.
Not even that, perhaps - the nature of caching needed is very different.
> Forgive me for stating what may be obvious to everyone else but, Ideally,
> I'd like to see three modules:
>
> A caching module with a generic api, such that it folks can write drop in
> replacements for the standard shipped module. People could then also use
> this module in any other module that could benefit from it, caching cgi,
> caching internal redirect, blah blah blah.
> A gateway module, just the relatively small bit of code to do proxypassing.
> A proxy module, most of the current functionality, minus the cache and
> proxypass code.
>
> Not only do I think that this makes much more sense then the current
> monolithic module, but I think that it would be significantly easier to
> find a maintainer for each smaller module.
I suspect a better solution would be a very simple, standalone daemon
acting as an "http router" - just accept connections and proxy the data
to/from a backend server (or a cache, perhaps) based on the request's
nature (source IP, URL requested, backend load levels, whatever). I'm
actually writing something which could do this anyway (in the very early
stages, though!).
Plan:
Various policies available to decide how to handle each request:
* Backend server load
* Source IP - redirect to a nearer mirror, perhaps
* URL requested - separate static and CGI requests
* Cache - if we've got a suitable cached item, send that
Thoughts??
James.
Re: [mod_proxy] Help offered
Posted by Sander van Zoest <sa...@covalent.net>.
On Wed, 7 Feb 2001 rbb@covalent.net wrote:
>> A gateway module, just the relatively small bit of code to do proxypassing.
>> A proxy module, most of the current functionality, minus the cache and
>> proxypass code.
> A proxy and a reverse proxy are incredibly similar things. The biggest
> difference is what is the machine that the proxy is sending the data
> to. In one case, that is found in the request itself, in the other it is
> in configuration data. These two functions really should be a single
> module where a good portion of the code is shared.
The biggest issue would be things like keep alive and upgrading and
downgrading of requests between 1.1 and 1.0.
The latency changes between the two as well. In the accelerator mode
The latency is in the initial request, where in the normal proxy mode
it is the request to the origin server.
it's really CDNP vs. WREC/WEBI here.
> Take a look at the definition of a gateway in RFC2616, it is basically
> saying, as I read it, that this is a proxy that looks to the outside world
> like an origin server.
Cheers,
--
Sander van Zoest [sander@covalent.net]
Covalent Technologies, Inc. http://www.covalent.net/
(415) 536-5218 http://www.vanzoest.com/sander/
Re: [mod_proxy] Help offered
Posted by Sander van Zoest <sa...@covalent.net>.
On Wed, 7 Feb 2001 rbb@covalent.net wrote:
>> A gateway module, just the relatively small bit of code to do proxypassing.
>> A proxy module, most of the current functionality, minus the cache and
>> proxypass code.
> A proxy and a reverse proxy are incredibly similar things. The biggest
> difference is what is the machine that the proxy is sending the data
> to. In one case, that is found in the request itself, in the other it is
> in configuration data. These two functions really should be a single
> module where a good portion of the code is shared.
The biggest issue would be things like keep alive and upgrading and
downgrading of requests between 1.1 and 1.0.
The latency changes between the two as well. In the accelerator mode
The latency is in the initial request, where in the normal proxy mode
it is the request to the origin server.
it's really CDNP vs. WREC/WEBI here.
> Take a look at the definition of a gateway in RFC2616, it is basically
> saying, as I read it, that this is a proxy that looks to the outside world
> like an origin server.
Cheers,
--
Sander van Zoest [sander@covalent.net]
Covalent Technologies, Inc. http://www.covalent.net/
(415) 536-5218 http://www.vanzoest.com/sander/
Re: [mod_proxy] Help offered
Posted by rb...@covalent.net.
> Where they do share common code functionality, it's often not totally
> perfect. Looking at the code I see no compelling reason to have one code
> base for these two functions. Caching is the one obvious section that is
> clearly sharable by the two sections.
>
> Forgive me for stating what may be obvious to everyone else but, Ideally,
> I'd like to see three modules:
>
> A caching module with a generic api, such that it folks can write drop in
> replacements for the standard shipped module. People could then also use
> this module in any other module that could benefit from it, caching cgi,
> caching internal redirect, blah blah blah.
This already exists. It needs to be bulked up, and made to cache
correctly, but I suggest looking at mod_cache in
modules/experimental. What is there is a proof of concept, but it
shouldn't be hard to expand upon.
> A gateway module, just the relatively small bit of code to do proxypassing.
> A proxy module, most of the current functionality, minus the cache and
> proxypass code.
A proxy and a reverse proxy are incredibly similar things. The biggest
difference is what is the machine that the proxy is sending the data
to. In one case, that is found in the request itself, in the other it is
in configuration data. These two functions really should be a single
module where a good portion of the code is shared.
Take a look at the definition of a gateway in RFC2616, it is basically
saying, as I read it, that this is a proxy that looks to the outside world
like an origin server.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: [mod_proxy] Help offered
Posted by rb...@covalent.net.
> Where they do share common code functionality, it's often not totally
> perfect. Looking at the code I see no compelling reason to have one code
> base for these two functions. Caching is the one obvious section that is
> clearly sharable by the two sections.
>
> Forgive me for stating what may be obvious to everyone else but, Ideally,
> I'd like to see three modules:
>
> A caching module with a generic api, such that it folks can write drop in
> replacements for the standard shipped module. People could then also use
> this module in any other module that could benefit from it, caching cgi,
> caching internal redirect, blah blah blah.
This already exists. It needs to be bulked up, and made to cache
correctly, but I suggest looking at mod_cache in
modules/experimental. What is there is a proof of concept, but it
shouldn't be hard to expand upon.
> A gateway module, just the relatively small bit of code to do proxypassing.
> A proxy module, most of the current functionality, minus the cache and
> proxypass code.
A proxy and a reverse proxy are incredibly similar things. The biggest
difference is what is the machine that the proxy is sending the data
to. In one case, that is found in the request itself, in the other it is
in configuration data. These two functions really should be a single
module where a good portion of the code is shared.
Take a look at the definition of a gateway in RFC2616, it is basically
saying, as I read it, that this is a proxy that looks to the outside world
like an origin server.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: [mod_proxy] Help offered
Posted by James Sutherland <ja...@cam.ac.uk>.
On Wed, 7 Feb 2001, Gabriel Russell wrote:
> At 01:11 PM 2/7/2001 +0000, James Sutherland wrote:
Eh? The next quote was not mine!
> > > > There 2 jobs share some common code/functionnality but there are also
> > quite
> > > > different.
>
> Where they do share common code functionality, it's often not totally
> perfect. Looking at the code I see no compelling reason to have one code
> base for these two functions. Caching is the one obvious section that is
> clearly sharable by the two sections.
Not even that, perhaps - the nature of caching needed is very different.
> Forgive me for stating what may be obvious to everyone else but, Ideally,
> I'd like to see three modules:
>
> A caching module with a generic api, such that it folks can write drop in
> replacements for the standard shipped module. People could then also use
> this module in any other module that could benefit from it, caching cgi,
> caching internal redirect, blah blah blah.
> A gateway module, just the relatively small bit of code to do proxypassing.
> A proxy module, most of the current functionality, minus the cache and
> proxypass code.
>
> Not only do I think that this makes much more sense then the current
> monolithic module, but I think that it would be significantly easier to
> find a maintainer for each smaller module.
I suspect a better solution would be a very simple, standalone daemon
acting as an "http router" - just accept connections and proxy the data
to/from a backend server (or a cache, perhaps) based on the request's
nature (source IP, URL requested, backend load levels, whatever). I'm
actually writing something which could do this anyway (in the very early
stages, though!).
Plan:
Various policies available to decide how to handle each request:
* Backend server load
* Source IP - redirect to a nearer mirror, perhaps
* URL requested - separate static and CGI requests
* Cache - if we've got a suitable cached item, send that
Thoughts??
James.
Re: [mod_proxy] Help offered
Posted by Gabriel Russell <g....@ieee.org>.
At 01:11 PM 2/7/2001 +0000, James Sutherland wrote:
> > > There 2 jobs share some common code/functionnality but there are also
> quite
> > > different.
Where they do share common code functionality, it's often not totally
perfect. Looking at the code I see no compelling reason to have one code
base for these two functions. Caching is the one obvious section that is
clearly sharable by the two sections.
Forgive me for stating what may be obvious to everyone else but, Ideally,
I'd like to see three modules:
A caching module with a generic api, such that it folks can write drop in
replacements for the standard shipped module. People could then also use
this module in any other module that could benefit from it, caching cgi,
caching internal redirect, blah blah blah.
A gateway module, just the relatively small bit of code to do proxypassing.
A proxy module, most of the current functionality, minus the cache and
proxypass code.
Not only do I think that this makes much more sense then the current
monolithic module, but I think that it would be significantly easier to
find a maintainer for each smaller module.
Re: [mod_proxy] Help offered
Posted by Gabriel Russell <g....@ieee.org>.
At 01:11 PM 2/7/2001 +0000, James Sutherland wrote:
> > > There 2 jobs share some common code/functionnality but there are also
> quite
> > > different.
Where they do share common code functionality, it's often not totally
perfect. Looking at the code I see no compelling reason to have one code
base for these two functions. Caching is the one obvious section that is
clearly sharable by the two sections.
Forgive me for stating what may be obvious to everyone else but, Ideally,
I'd like to see three modules:
A caching module with a generic api, such that it folks can write drop in
replacements for the standard shipped module. People could then also use
this module in any other module that could benefit from it, caching cgi,
caching internal redirect, blah blah blah.
A gateway module, just the relatively small bit of code to do proxypassing.
A proxy module, most of the current functionality, minus the cache and
proxypass code.
Not only do I think that this makes much more sense then the current
monolithic module, but I think that it would be significantly easier to
find a maintainer for each smaller module.
Re: [mod_proxy] Help offered
Posted by James Sutherland <ja...@cam.ac.uk>.
On Tue, 6 Feb 2001, Sander van Zoest wrote:
> On Tue, 6 Feb 2001, Philippe M . Chiasson wrote:
>
> > I know of at least 2 very different uses for the mod_proxy module currently.
> > The first one is to act as a caching HTTP proxy, but the second is to make a
> > lightweight doc server proxy some requests to a heavyweight server (mod_perl).
> > There 2 jobs share some common code/functionnality but there are also quite
> > different.
> >
> > I think it would be wise to consider splitting these 2 tasks up to.
>
> Although it should considered, I think the power of an apache proxy server
> would be the fact that it could do both of these tasks. I would consider
> a proxy type of project to be more along the lines of what Dean Gaudet has
> been refering to as an "http router". Everyone is doing this one way or
> another and this is also part of squid now (with their accelerated support).
If the meaning is what I think it is, that would definitely be a good
project. You mean something akin to Squid's "httpd acceleration" mode -
you accept the connections, parse them to get the URL, then pass the
request on to a suitable backend? (Or serve static content from a cache.)
> If some people add a reliable cache that talks HTCP and ICP and whatever they
> feel is useful that would be a bonus. Having access via different languages to
> the proxy/router stream can be very beneficial as well.
>
> Maybe if the project would come out with that sort of scope more people would
> be interested in helping out. I would definately use it and submit patches
> if I can not find the time to work on it in a fuller capacity.
I'd certainly like to help on that...
James.
Re: [mod_proxy] Help offered
Posted by James Sutherland <ja...@cam.ac.uk>.
On Tue, 6 Feb 2001, Sander van Zoest wrote:
> On Tue, 6 Feb 2001, Philippe M . Chiasson wrote:
>
> > I know of at least 2 very different uses for the mod_proxy module currently.
> > The first one is to act as a caching HTTP proxy, but the second is to make a
> > lightweight doc server proxy some requests to a heavyweight server (mod_perl).
> > There 2 jobs share some common code/functionnality but there are also quite
> > different.
> >
> > I think it would be wise to consider splitting these 2 tasks up to.
>
> Although it should considered, I think the power of an apache proxy server
> would be the fact that it could do both of these tasks. I would consider
> a proxy type of project to be more along the lines of what Dean Gaudet has
> been refering to as an "http router". Everyone is doing this one way or
> another and this is also part of squid now (with their accelerated support).
If the meaning is what I think it is, that would definitely be a good
project. You mean something akin to Squid's "httpd acceleration" mode -
you accept the connections, parse them to get the URL, then pass the
request on to a suitable backend? (Or serve static content from a cache.)
> If some people add a reliable cache that talks HTCP and ICP and whatever they
> feel is useful that would be a bonus. Having access via different languages to
> the proxy/router stream can be very beneficial as well.
>
> Maybe if the project would come out with that sort of scope more people would
> be interested in helping out. I would definately use it and submit patches
> if I can not find the time to work on it in a fuller capacity.
I'd certainly like to help on that...
James.
Re: [mod_proxy] Help offered
Posted by Sander van Zoest <sa...@covalent.net>.
On Tue, 6 Feb 2001, Philippe M . Chiasson wrote:
> I know of at least 2 very different uses for the mod_proxy module currently.
> The first one is to act as a caching HTTP proxy, but the second is to make a
> lightweight doc server proxy some requests to a heavyweight server (mod_perl).
> There 2 jobs share some common code/functionnality but there are also quite
> different.
>
> I think it would be wise to consider splitting these 2 tasks up to.
Although it should considered, I think the power of an apache proxy server
would be the fact that it could do both of these tasks. I would consider
a proxy type of project to be more along the lines of what Dean Gaudet has
been refering to as an "http router". Everyone is doing this one way or
another and this is also part of squid now (with their accelerated support).
If some people add a reliable cache that talks HTCP and ICP and whatever they
feel is useful that would be a bonus. Having access via different languages to
the proxy/router stream can be very beneficial as well.
Maybe if the project would come out with that sort of scope more people would
be interested in helping out. I would definately use it and submit patches
if I can not find the time to work on it in a fuller capacity.
--
Sander van Zoest [sander@covalent.net]
Covalent Technologies, Inc. http://www.covalent.net/
(415) 536-5218 http://www.vanzoest.com/sander/
Re: [mod_proxy] Help offered
Posted by Sander van Zoest <sa...@covalent.net>.
On Tue, 6 Feb 2001, Philippe M . Chiasson wrote:
> I know of at least 2 very different uses for the mod_proxy module currently.
> The first one is to act as a caching HTTP proxy, but the second is to make a
> lightweight doc server proxy some requests to a heavyweight server (mod_perl).
> There 2 jobs share some common code/functionnality but there are also quite
> different.
>
> I think it would be wise to consider splitting these 2 tasks up to.
Although it should considered, I think the power of an apache proxy server
would be the fact that it could do both of these tasks. I would consider
a proxy type of project to be more along the lines of what Dean Gaudet has
been refering to as an "http router". Everyone is doing this one way or
another and this is also part of squid now (with their accelerated support).
If some people add a reliable cache that talks HTCP and ICP and whatever they
feel is useful that would be a bonus. Having access via different languages to
the proxy/router stream can be very beneficial as well.
Maybe if the project would come out with that sort of scope more people would
be interested in helping out. I would definately use it and submit patches
if I can not find the time to work on it in a fuller capacity.
--
Sander van Zoest [sander@covalent.net]
Covalent Technologies, Inc. http://www.covalent.net/
(415) 536-5218 http://www.vanzoest.com/sander/