How to proxy multiple web servers?

[Brijesh Deo <bd...@SonicWALL.com>]

Hi,

 

I am implementing a Reverse proxy application modeled on the sample code
'ElementalReverseProxy'
(http://svn.apache.org/repos/asf/httpcomponents/httpcore/trunk/httpcore/
src/examples/org/apache/http/examples/ElementalReverseProxy.java) with a
little modification. 

 

My Reverse Proxy application has an HttpService running on a port for
handling incoming HttpRequests. For, passing on these Requests to the
Target web servers (for outgoing connections), I am using HttpClient
4.0. This is what I have done differently from the
ElementalReverseProxy.java. The HttpResponse thus obtained from the
Target webservers is passed into the Response to the initial Request
that was received by the HttpService. So, this works great as a reverse
proxy.

 

My Reverse proxy application needs to proxy multiple web servers and
such web servers can be dynamically added or deleted from the proxy. The
above approach works great when I dedicate a HttpService on a specific
port to proxy a particular web server only. So, going by the above
approach, I would need to run multiple HttpServices on different ports
to proxy each web server. And that many ports have to be left open on
the system on which I deploy this application. So, this poses a security
risk and also can be a scalability issue when I need to proxy hundreds
of web servers. Actually, all my webservers are small network appliances
that run a web server with the management web GUI which I want to proxy
through a centralized system.

 

The ideal solution I believe would be to run a single HttpService on a
port and then send the Requests to different target hosts using the
Httpclient, but the challenge is how to find out by seeing the
HttpRequest that to which target it needs to be sent. Is it good to set
a Cookie in the Response to the first request that comes in with an
identifier for a particular web server? I could have the first request
in the form :
http://my-reverseproxy-host:8888/index.html?id=some_identifier. In the
response to this request, I could add the Set-Cookie header with the
identifier value which should help me identify subsequent request for
that browser session.

 

What are the other ways to keep track of the Requests and responses? Has
anyone come across this kind of situation?

 

Please let me know.

 

Thanks,

Brijesh

 

Re: How to proxy multiple web servers?

[Oleg Kalnichevski <ol...@apache.org>]

On Wed, Apr 08, 2009 at 09:24:21AM +0200, Ortwin Gl?ck wrote:
> Brijesh Deo wrote:
> > So, going by the above
> > approach, I would need to run multiple HttpServices on different ports
> > to proxy each web server. 
> 
> Brijesh,
> 
> Why don't you use the Host header for multiplexing? That's how evey shared
> webhosting out there works.
> 

Spot on.

Oleg

> Cheers,
> 
> Ortwin
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org

Re: How to proxy multiple web servers?

[Ortwin Glück <od...@odi.ch>]

Brijesh Deo wrote:
> So, going by the above
> approach, I would need to run multiple HttpServices on different ports
> to proxy each web server. 

Brijesh,

Why don't you use the Host header for multiplexing? That's how evey shared
webhosting out there works.

Cheers,

Ortwin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org

Re: How to proxy multiple web servers?

[Sam Crawford <sa...@gmail.com>]

Hi Brijesh,

Just so I'm clear, your challenge is working out which backend host to send
the requests to based upon the incoming client request?

The way I see it you've got a few options:

1. Use name based virtual hosting.So you'd have loads of sub-domains like
device001.company.com, device002.company.com, that all point to your reverse
proxy server (how you do it in DNS is up to you - maybe a wildcard A record,
or specific CNAMES, who knows). Then when your web browser visits
device006.company.com it'll send a "Host: device006.company.com" HTTP header
to your reverse proxy, and you can use this to decide which backend to send
it to (Granted this will only work for HTTP/1.1 clients, so if you need to
support HTTP/1.0 - and I can't see why - then you might have to use
something else).

2. If you can't or do not want to use name based virtual hosting then how
much configuration can you do on the backend devices? Can you run their
webservers all under a different context-root? e.g. device001 would run
under http://10.0.0.1/device001/ and device002 would run under
http://10.0.0.2/device002/ etc. If you could modify the context-roots of
your backend servers then you could use the first part of the URI in the
incoming request to determine the backend.

3. Failing the above, you could use some kind of query string check along
with cookie setting. I'd be very very wary of doing this personally, as it
essentially limits you to working on one backend host per browser session.
But anyway, the process would be something like:
(a) Client requests /index.html?id=device001 from reverse proxy host
(b) Reverse proxy looks up device001 in table, finds that it's valid
(c) Reverse proxy notes that you haven't passed in a session cookie and
builds you a new session with "id=device001" as a attribute/value pair
(d) Reverse proxy sends your request to the backend (device001)
(e) Reverse proxy returns the response to you, along with the "Set-Cookie"
header so that your browser gets a cookie
(f) Client requests /device_stats.html, but this time passes in the session
cookie. Reverse proxy looks this up in memory and proceeds from step (d)

I'd stick with #1 or #2 personally.

Hope this helps,

Sam




2009/4/8 Brijesh Deo <bd...@sonicwall.com>

> Hi,
>
>
>
> I am implementing a Reverse proxy application modeled on the sample code
> 'ElementalReverseProxy'
> (http://svn.apache.org/repos/asf/httpcomponents/httpcore/trunk/httpcore/
> src/examples/org/apache/http/examples/ElementalReverseProxy.java<http://svn.apache.org/repos/asf/httpcomponents/httpcore/trunk/httpcore/%0Asrc/examples/org/apache/http/examples/ElementalReverseProxy.java>)
> with a
> little modification.
>
>
>
> My Reverse Proxy application has an HttpService running on a port for
> handling incoming HttpRequests. For, passing on these Requests to the
> Target web servers (for outgoing connections), I am using HttpClient
> 4.0. This is what I have done differently from the
> ElementalReverseProxy.java. The HttpResponse thus obtained from the
> Target webservers is passed into the Response to the initial Request
> that was received by the HttpService. So, this works great as a reverse
> proxy.
>
>
>
> My Reverse proxy application needs to proxy multiple web servers and
> such web servers can be dynamically added or deleted from the proxy. The
> above approach works great when I dedicate a HttpService on a specific
> port to proxy a particular web server only. So, going by the above
> approach, I would need to run multiple HttpServices on different ports
> to proxy each web server. And that many ports have to be left open on
> the system on which I deploy this application. So, this poses a security
> risk and also can be a scalability issue when I need to proxy hundreds
> of web servers. Actually, all my webservers are small network appliances
> that run a web server with the management web GUI which I want to proxy
> through a centralized system.
>
>
>
> The ideal solution I believe would be to run a single HttpService on a
> port and then send the Requests to different target hosts using the
> Httpclient, but the challenge is how to find out by seeing the
> HttpRequest that to which target it needs to be sent. Is it good to set
> a Cookie in the Response to the first request that comes in with an
> identifier for a particular web server? I could have the first request
> in the form :
> http://my-reverseproxy-host:8888/index.html?id=some_identifier. In the
> response to this request, I could add the Set-Cookie header with the
> identifier value which should help me identify subsequent request for
> that browser session.
>
>
>
> What are the other ways to keep track of the Requests and responses? Has
> anyone come across this kind of situation?
>
>
>
> Please let me know.
>
>
>
> Thanks,
>
> Brijesh
>
>
>
>