You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Dave Rolsky <au...@urth.org> on 2000/11/27 01:02:41 UTC
Proxy adding headers
This sort of a mod_perl question. When I use mod_rewrite to proxy to my
mod_perl backend servers, it seems that even if I explicitly don't send
headers I still get something like:
HTTP/1.0 OK
Date: blah bah
in front of anything I said. The issue here is that I wrote some MP3
serving code based on Lincoln Stein's Apache::MP3 module. To send
streaming mp3 data to a client like Winamp or XMMS, I need to be able to
have absolute control over the headers that are generated.
Unfortunately, it seems that when I proxy I get the above added so I have
to serve streaming audio directly from the mod_perl server, which is less
than ideal in terms of resource use.
Any ideas?
-dave
/*==================
www.urth.org
We await the New Sun
==================*/
Re: More Speed -> mod_perl Module for HTML Compression
Posted by Robin Berjon <ro...@knowscape.com>.
At 18:14 30/11/2000 +0000, Nigel Hamilton wrote:
> Also for average-sized files, does the time taken to perform the
>decompression/compression negate any speed increase gained by reduced file
>size?
I don't have numbers to back this, but I'm ready to bet that writing an
uncompression algo meant to run within a browser is going to be *slow*.
Why do you want to do this ? I can't see any reason for wanting this
instead of using gz encoding, which will be faster by much, and is a well
proven method of sending large files faster.
-- robin b.
Learn from your parents mistakes - use birth control.
Re: Proxy adding headers
Posted by Dave Rolsky <au...@urth.org>.
On Tue, 28 Nov 2000, Gunther Birznieks wrote:
> Dave's company could also *pay* someone to do what he wants. It would
> probably take about a day of someone at Covalent (probably less) to whip
> something up to stop doing the headers (and they would probably be able to
> feed the change back into the mod_proxy part of the Apache CVS directly so
> others would benefit).
Great, as soon as I can find a company willing to fund my work on a
personal MP3 server I'll be all set ;)
Actually, I already did most of the patch and I sent a message to the
new-httpd list to discuss the possibility of making this a part of a
future release. Of course, if someone at Covalent wants to help that's
great too. And for the record, it took a lot less than a day. About
1.5-2 hours, I'd say.
-dave
/*==================
www.urth.org
We await the New Sun
==================*/
Re: Proxy adding headers
Posted by Gunther Birznieks <gu...@extropia.com>.
Dave's company could also *pay* someone to do what he wants. It would
probably take about a day of someone at Covalent (probably less) to whip
something up to stop doing the headers (and they would probably be able to
feed the change back into the mod_proxy part of the Apache CVS directly so
others would benefit).
At 10:10 PM 11/27/2000 -0500, Joe Schaefer wrote:
>Dave Rolsky <au...@urth.org> writes:
>
> > I'm afraid of C.
>
>Don't be. (perl is C with cream and sugar added ;) Grab
>a copy of Kernighan and Ritchie's _The C Programming Language_
>(it's authoritative, and all of 274 pages cover to cover).
>Then start monkeying around in src/modules/proxy/proxy_http.c
>and/or src/main/http.protocol.c and see what you can cook up.
>Tweaking and/or commenting out a few lines should do the trick.
>Just be sure to back up your apache tree before you start tinkering.
>
>Best.
>--
>Joe Schaefer
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: modperl-unsubscribe@apache.org
>For additional commands, e-mail: modperl-help@apache.org
__________________________________________________
Gunther Birznieks (gunther.birznieks@extropia.com)
eXtropia - The Web Technology Company
http://www.extropia.com/
Re: Proxy adding headers
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Dave Rolsky <au...@urth.org> writes:
> I'm afraid of C.
Don't be. (perl is C with cream and sugar added ;) Grab
a copy of Kernighan and Ritchie's _The C Programming Language_
(it's authoritative, and all of 274 pages cover to cover).
Then start monkeying around in src/modules/proxy/proxy_http.c
and/or src/main/http.protocol.c and see what you can cook up.
Tweaking and/or commenting out a few lines should do the trick.
Just be sure to back up your apache tree before you start tinkering.
Best.
--
Joe Schaefer
Re: Proxy adding headers
Posted by Dave Rolsky <au...@urth.org>.
On 27 Nov 2000, Joe Schaefer wrote:
> mod_proxy will upgrade assbackwards requests to HTTP/1.0
> before passing them along to the backend server, which may
> explain why the date field shows up in your telnet experiments.
> Why not post the full output of your telnet sessions so we
> can see what is really going on?
Actually, this totally explains the problem, as did Sander's previous
post. I'm attempting to use Apache as a non-HTTP server (sending ICY
headers for streaming MP3 instead). So yes, its an assbackwards request
and its upgrading it. Which is annoying. It would be nice if this were
configurable in mod_proxy.c. Maybe I will have to look at the code. But
I'm afraid of C.
-dave
/*==================
www.urth.org
We await the New Sun
==================*/
Re: Proxy adding headers
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Dave Rolsky <au...@urth.org> writes:
> On Mon, 27 Nov 2000, Sander van Zoest wrote:
>
> > Normally the mod_proxy code doesn't touch the headers, it simply sends
> > the headers on from the remote server you are proxying too. I doubt
> > it is mod_rewrite but it could be. The extra headers are probably coming
> > from your remote server you are proxying to.
>
> I don't think so. If I do a telnet to the proxy server (port 80, no
> mod_perl) I get the extra headers. If I telnet to the mod_perl enabled
> server (port 12345), I get what I want (no extra headers).
>
Technically mod_proxy doesn't generate headers (here I'm
taking the simple case of a non-cacheing http request).
After stripping the connection header, it simply passes
the headers it got from the backend server right on along
to r->connection via ap_proxy_send_hdr_line (or
ap_proxy_send_headers for cached files IIRC).
Moreover if your backend server calls ap_send_http_header
during its response, it *must* return a Server and Date field
unless your request is assbackwards. See the src for
ap_basic_http_header in src/main/http_protocol.c.
mod_proxy will upgrade assbackwards requests to HTTP/1.0
before passing them along to the backend server, which may
explain why the date field shows up in your telnet experiments.
Why not post the full output of your telnet sessions so we
can see what is really going on?
HTH.
--
Joe Schaefer
Re: [OT] Re: Proxy adding headers
Posted by Dave Rolsky <au...@urth.org>.
On Mon, 27 Nov 2000, Sander van Zoest wrote:
> Ah, are you trying to send ICY headers or something? mod_proxy only knows
> of HTTP and sends the appropriate status itself rather then what it
> gets from the remote server. This will also require some hacks in
> mod_proxy to make it aware of the protocol you are trying to proxy.
That is the exact problem. Ok, the chances of me hacking mod_proxy are
next to nil. At least now I know what I can't do to fix the problem
though ;)
-dave
/*==================
www.urth.org
We await the New Sun
==================*/
Re: [OT] Re: Proxy adding headers
Posted by Dave Rolsky <au...@urth.org>.
On Mon, 27 Nov 2000, Sander van Zoest wrote:
> Ah, are you trying to send ICY headers or something? mod_proxy only knows
> of HTTP and sends the appropriate status itself rather then what it
> gets from the remote server. This will also require some hacks in
> mod_proxy to make it aware of the protocol you are trying to proxy.
Well, I guess C is easier than I thought. I got it working via a new
config directive for mod_proxy, ProxyAddHeaders. By default its true and
this means that mod_proxy will add minimal headers for wack responses from
the proxy server (like HTTP 0.9 or ICY or whatever). However, if you turn
it off it will _not_ add these headers. There is a bug in the current
state of the code in that if the caching code is called, it will add a
Date header if none exists. I'm currently working around this by turning
off caching.
I'm going to see if there is interest in the Apache core team in having
this submitted as a patch. Is so, I'll work around the remaining bug.
In the meantime, if anybody on this list is interested I can send you my
current patch, which is against 1.3.12. Its not many lines so porting it
to a different Apache version should be trivial.
-dave
/*==================
www.urth.org
We await the New Sun
==================*/
[OT] Re: Proxy adding headers
Posted by Sander van Zoest <sa...@covalent.net>.
On Mon, 27 Nov 2000, Dave Rolsky wrote:
> On Mon, 27 Nov 2000, Sander van Zoest wrote:
> > Normally the mod_proxy code doesn't touch the headers, it simply sends
> > the headers on from the remote server you are proxying too. I doubt
> > it is mod_rewrite but it could be. The extra headers are probably coming
> > from your remote server you are proxying to.
> I don't think so. If I do a telnet to the proxy server (port 80, no
> mod_perl) I get the extra headers. If I telnet to the mod_perl enabled
> server (port 12345), I get what I want (no extra headers).
Ah, are you trying to send ICY headers or something? mod_proxy only knows
of HTTP and sends the appropriate status itself rather then what it
gets from the remote server. This will also require some hacks in
mod_proxy to make it aware of the protocol you are trying to proxy.
Cheers,
--
Sander van Zoest [sander@covalent.net]
Covalent Technologies, Inc. http://www.covalent.net/
(415) 536-5218 http://www.vanzoest.com/sander/
Re: Proxy adding headers
Posted by Dave Rolsky <au...@urth.org>.
On Mon, 27 Nov 2000, Sander van Zoest wrote:
> Normally the mod_proxy code doesn't touch the headers, it simply sends
> the headers on from the remote server you are proxying too. I doubt
> it is mod_rewrite but it could be. The extra headers are probably coming
> from your remote server you are proxying to.
I don't think so. If I do a telnet to the proxy server (port 80, no
mod_perl) I get the extra headers. If I telnet to the mod_perl enabled
server (port 12345), I get what I want (no extra headers).
-dave
/*==================
www.urth.org
We await the New Sun
==================*/
Re: Proxy adding headers
Posted by Sander van Zoest <sa...@covalent.net>.
On Mon, 27 Nov 2000, Dave Rolsky wrote:
> Actually, something is _adding_ headers and I want it to stop doing it.
> It may not be mod_proxy, perhaps it is mod_rewrite. I really don't know.
Normally the mod_proxy code doesn't touch the headers, it simply sends
the headers on from the remote server you are proxying too. I doubt
it is mod_rewrite but it could be. The extra headers are probably coming
from your remote server you are proxying to.
--
Sander van Zoest [sander@covalent.net]
Covalent Technologies, Inc. http://www.covalent.net/
(415) 536-5218 http://www.vanzoest.com/sander/
Re: Proxy adding headers
Posted by Dave Rolsky <au...@urth.org>.
On Mon, 27 Nov 2000, Sander van Zoest wrote:
> > Unfortunately, it seems that when I proxy I get the above added so I have
> > to serve streaming audio directly from the mod_perl server, which is less
> > than ideal in terms of resource use.
>
> If you mena you rewrite to mod_proxy, then mod_proxy does not fill
> headers{in} it simply ships what it gets from the remote server with the
> content and ships that. You would need to hack mod_proxy for it parse
> the headers and provide them to you in the normal request_rec structures.
>
> I have don't this before, but do not have rights to the code to post the
> patch, sorry. :-(
Actually, something is _adding_ headers and I want it to stop doing it.
It may not be mod_proxy, perhaps it is mod_rewrite. I really don't know.
-dave
/*==================
www.urth.org
We await the New Sun
==================*/
Re: Proxy adding headers
Posted by Sander van Zoest <sa...@covalent.net>.
On Sun, 26 Nov 2000, Dave Rolsky wrote:
> This sort of a mod_perl question. When I use mod_rewrite to proxy to my
> mod_perl backend servers, it seems that even if I explicitly don't send
> headers I still get something like:
>
> Unfortunately, it seems that when I proxy I get the above added so I have
> to serve streaming audio directly from the mod_perl server, which is less
> than ideal in terms of resource use.
If you mena you rewrite to mod_proxy, then mod_proxy does not fill
headers{in} it simply ships what it gets from the remote server with the
content and ships that. You would need to hack mod_proxy for it parse
the headers and provide them to you in the normal request_rec structures.
I have don't this before, but do not have rights to the code to post the
patch, sorry. :-(
Hope that helps,
--
Sander van Zoest [sander@covalent.net]
Covalent Technologies, Inc. http://www.covalent.net/
(415) 536-5218 http://www.vanzoest.com/sander/
Re: More Speed -> mod_perl Module for HTML Compression
Posted by Joshua Chamas <jo...@chamas.com>.
Nigel Hamilton wrote:
>
> Does anyone know of a mod_perl module that compresses HTML and a
> companion Javascript procedure that decompresses the data on the
> client-side?
>
> I know there are Gzip modules that zip files on the way back to
> the browser ... but I'm after something that zips on the server and
> decompresses transparently in Javascript across all browsers. Ideally I
> want to do: document.write(uncompressed-contents) in Javascript on the
> client-side.
>
To add to Matt's comments and likely Ged would agree, you'll probably
find that Gzip compression is better supported cross browser than
any JavaScript you could come up with. JavaScript breaks in lots of
ways all the time if you just look at IE4-IE5, NS4.0x-NS4.7x. And then
look at them on NT/2000 vs. 95/98/ME, that'll really kill ya.
--Josh
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks >> free web link monitoring Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Re: More Speed -> mod_perl Module for HTML Compression
Posted by "Randal L. Schwartz" <me...@stonehenge.com>.
>>>>> "Ken" == Ken Williams <ke...@forum.swarthmore.edu> writes:
Ken> In particular, it seems like you think that users have to manually
Ken> decompress gzipped content, but that's not the case. Just thought I'd
Ken> state it if that was the confusion.
Ken> mod_gzip, Apache::Compress, or Apache::Gzip are solutions here.
Or even my cool compressing pre-forking tiny proxy at
<http://www.stonehenge.com/merlyn/WebTechniques/col34.html>
Neatly proxies, but sends compressed text across slow links if
the browser understands that.
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<me...@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
Re: More Speed -> mod_perl Module for HTML Compression
Posted by Ken Williams <ke...@forum.swarthmore.edu>.
nigel@e1mail.com (Nigel Hamilton) wrote:
> I'm trying to reduce the amount of data sent from server to
>browser by using compression ---> hopefully accelerating the time to
>serve a page.
>
> Does anyone know of a mod_perl module that compresses HTML and a
>companion Javascript procedure that decompresses the data on the
>client-side?
>
> I know there are Gzip modules that zip files on the way back to
>the browser ... but I'm after something that zips on the server and
>decompresses transparently in Javascript across all browsers. Ideally I
>want to do: document.write(uncompressed-contents) in Javascript on the
>client-side.
I think you've got a slight misconception about how gzip HTTP
compression works. It's perfectly transparent, in that browsers that
support compression will decompress the file automatically, and the user
will never know that the page was compressed in the first place. That's
much smoother than the javascript decompression you propose, which I
can't help thinking will turn into a real headache, perhaps even a
nightmare.
In particular, it seems like you think that users have to manually
decompress gzipped content, but that's not the case. Just thought I'd
state it if that was the confusion.
mod_gzip, Apache::Compress, or Apache::Gzip are solutions here.
------------------- -------------------
Ken Williams Last Bastion of Euclidity
ken@forum.swarthmore.edu The Math Forum
Re: More Speed -> mod_perl Module for HTML Compression
Posted by Matt Sergeant <ma...@sergeant.org>.
On Thu, 30 Nov 2000, Nigel Hamilton wrote:
> Hi,
> I'm trying to reduce the amount of data sent from server to
> browser by using compression ---> hopefully accelerating the time to
> serve a page.
>
> Does anyone know of a mod_perl module that compresses HTML and a
> companion Javascript procedure that decompresses the data on the
> client-side?
>
> I know there are Gzip modules that zip files on the way back to
> the browser ... but I'm after something that zips on the server and
> decompresses transparently in Javascript across all browsers. Ideally I
> want to do: document.write(uncompressed-contents) in Javascript on the
> client-side.
>
> Has anyone come up with something for this?
Nobody here would be mad enough to do this... Is it on an intranet? If
not, you'll never get me visiting your site - I don't enable javascript
generally.
> Also for average-sized files, does the time taken to perform the
> decompression/compression negate any speed increase gained by reduced file
> size?
I don't think so, but it probably depends a huge amount on the size of
your pipe and how many pages you're hoping to server. For example, I'm on
a 64K pipe, so CPU isn't the limiting factor of what I can serve - the
pipe is. So I gzip and can serve more pages.
--
<Matt/>
/|| ** Director and CTO **
//|| ** AxKit.com Ltd ** ** XML Application Serving **
// || ** http://axkit.org ** ** XSLT, XPathScript, XSP **
// \\| // ** Personal Web Site: http://sergeant.org/ **
\\//
//\\
// \\
More Speed -> mod_perl Module for HTML Compression
Posted by Nigel Hamilton <ni...@e1mail.com>.
Hi,
I'm trying to reduce the amount of data sent from server to
browser by using compression ---> hopefully accelerating the time to
serve a page.
Does anyone know of a mod_perl module that compresses HTML and a
companion Javascript procedure that decompresses the data on the
client-side?
I know there are Gzip modules that zip files on the way back to
the browser ... but I'm after something that zips on the server and
decompresses transparently in Javascript across all browsers. Ideally I
want to do: document.write(uncompressed-contents) in Javascript on the
client-side.
Has anyone come up with something for this?
Also for average-sized files, does the time taken to perform the
decompression/compression negate any speed increase gained by reduced file
size?
Nige
Nigel Hamilton
______________________________________________________________________________