[users@httpd] Reverse Proxying of multipart messages

[Michael Smith <sm...@gmail.com>]

Hi there,

I'm trying to set up reverse proxying to present a multipart document
successfully to the user.

For example, I have the following simple CGI script:

--
#!/usr/local/bin/perl

$| = 1;
$lines=20;
use CGI qw/:push -nph/;
print multipart_init(-boundary=>'----here we go!');
         foreach (0 .. $lines) {
             print multipart_start(-type=>'text/plain'),
                   "The current time is ",scalar(localtime),"\n";
             if ($_ < $lines) {
                     print multipart_end;
             } else {
                     print multipart_final;
             }
             sleep(1);
        }
--

When I connect to this server directly, the output appears a bit at a time.

When I have this server behind a proxy server (using mod_proxy in
apache 1.3 or 2.0), the output comes out in big chunks.

I've tried setting ProxyIOBufferSize and ProxyReceiveBufferSize to
small values in the hope that I can get the reverse proxy server to
output a character at a time instead of holding on to it and giving
the output in larger sizes.   However this didn't have any obvious
impact.

Ideally mod_proxy would recognise the message as multipart and treat
it accordingly, but if it just transmitted the data a byte at a time
that would also work for me.

Anyone any ideas on whether this should work, or how to make it work,
or what development would need to be done?

Thank you

Michael Smith

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: [users@httpd] Reverse Proxying of multipart messages

[Michael Smith <sm...@gmail.com>]

> > When I have this server behind a proxy server (using mod_proxy in
> > apache 1.3 or 2.0), the output comes out in big chunks.
> 
> Let me guess: for values of "big" in the region of 8Kb?

You got it ... great guess ;)
> 
> > I've tried setting ProxyIOBufferSize and ProxyReceiveBufferSize to
> 
> Won't help.  The proxy is designed to run efficiently.  The delays you are
> introducing are a bad mismatch with HTTP, so they'll only work with HTTP
> under benign "too simple to fail" conditions.  Proxy simply doesn't meet that,
> at least until your chunk size rises sufficiently above apache's default heap
> bucket size (8K) that buffering your data won't dominate.
> 
> > Ideally mod_proxy would recognise the message as multipart and treat
> > it accordingly, but if it just transmitted the data a byte at a time
> > that would also work for me.
> 
> It could be made to do either of those with a suitable module to parse your
> contents.  But in HTTP terms - which is all mod_proxy speaks out-of-the-box,
> it is NOT multipart.  It's a *single* HTTP response, whose *contents* (which
> the proxy won't unpack) happens to be MIMEd.
> 
> Kind-of like expecting the proxy to treat every entry in a .tar or .zip file
> as separate.  Lots of work, nothing to gain.  The whole point of tarring or
> zipping your files is to shift them efficiently, and that's what proxy does.

Very many thanks for the detailed response.  I understand things a lot
better now.

What I am trying to do is to serve an HTML page which is dynamically
updated via a multipart xmlhttp message.  I get javascript permission
errors if the server I try to connect to isn't the one that the
original HTML was retrieved from.  And seeing as I'm using reverse
proxying to load balance, it seems therefore that I have to go through
the proxy.

I have got this same sort of thing working with a java applet instead
of xmlhttp, as this does not seem to be bound by the same
restrictions.

Any ideas for the best way round this?

Thanks

Michael Smith

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: [users@httpd] Reverse Proxying of multipart messages

[Nick Kew <ni...@webthing.com>]

On Friday 04 March 2005 12:06, Michael Smith wrote:

> I'm trying to set up reverse proxying to present a multipart document
> successfully to the user.

That's not actually your problem.  What you're trying to do that isn't working
well is to introduce delays, under your own control, into an HTTP response.

> When I connect to this server directly, the output appears a bit at a time.
>
> When I have this server behind a proxy server (using mod_proxy in
> apache 1.3 or 2.0), the output comes out in big chunks.

Let me guess: for values of "big" in the region of 8Kb?

> I've tried setting ProxyIOBufferSize and ProxyReceiveBufferSize to

Won't help.  The proxy is designed to run efficiently.  The delays you are
introducing are a bad mismatch with HTTP, so they'll only work with HTTP
under benign "too simple to fail" conditions.  Proxy simply doesn't meet that,
at least until your chunk size rises sufficiently above apache's default heap
bucket size (8K) that buffering your data won't dominate.

> Ideally mod_proxy would recognise the message as multipart and treat
> it accordingly, but if it just transmitted the data a byte at a time
> that would also work for me.

It could be made to do either of those with a suitable module to parse your
contents.  But in HTTP terms - which is all mod_proxy speaks out-of-the-box, 
it is NOT multipart.  It's a *single* HTTP response, whose *contents* (which 
the proxy won't unpack) happens to be MIMEd.

Kind-of like expecting the proxy to treat every entry in a .tar or .zip file 
as separate.  Lots of work, nothing to gain.  The whole point of tarring or
zipping your files is to shift them efficiently, and that's what proxy does.


-- 
Nick Kew

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org