You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Rene Mayrhofer <re...@fh-hagenberg.at> on 2011/08/29 11:30:01 UTC
Proxying to dynamically changing origin servers
Hi everybody,
[Please CC me in replies, I am not currently subscribed to this mailing
list.]
For a research project, we currently have an interesting problem to
solve: to connect HTTPS clients to dynamically changing, internal HTTPS
servers. One approach that we are currently evaluating is to use a
publicly accessible HTTP proxy with CONNECT support to "tunnel" the
HTTPS connections to the internal servers. However, the internal
addresses may change dynamically. The question is therefore if Traffic
Server can be configured to (or if it is easy to write a plug-in to):
a) be used in "normal" proxy server mode for clients with explicit proxy
server configuration to use the CONNECT call for some HTTPS URLs; and
b) for the origin server resolving to be done dynamically based on
internal look-up tables. E.g. the URL
https://my-example.local.com/whatever specified by any client in the
CONNECT request should be mapped to host 10.20.30.40 (the HTTPS server
may use my-example.local.com as its server address, but the IP will
change dynamically).
I am aware that this is a mix between reverse proxy functionality
(mapping to internal servers) and normal proxying (CONNECT to
client-specified, different URLs). Based on the SDK documentation, I am
also unsure which kind of plug-in would be required to make this work.
The backup plan is to use a "normal" proxy with dynamic DNS for
resolving the internal IP addresses, but we would like to avoid this
complexity if possible.
Are we on the right track and is this possible with Traffic Server?
best regards,
Rene
RE: Serving Stale Cached Content when origin server unreachable
Posted by "Walsh, Peter" <Pe...@disney.com>.
Hello all,
I just wanted to follow up on my earlier email below here to see if someone can confirm my assumptions.
Thank you,
Pete Walsh
Software Engineer
206-664-4150
-----Original Message-----
From: Walsh, Peter [mailto:Peter.Walsh@disney.com]
Sent: Thursday, September 01, 2011 3:02 PM
To: users@trafficserver.apache.org
Subject: Serving Stale Cached Content when origin server unreachable
Hello all,
I found information in the Admin docs that indicate if the origin server doesn’t respond to a revalidation query, Traffic Server serves stale data along with a warning.
My question is: How often will Traffic Server try to revalidate the object when it has already been unable perform a revalidation query for that object before? My assumption is that Traffic Server will attempt to perform the revalidation on a single client request at a time and any other requests that come in for that same object will be served the stale cached object for as long as the time set in “proxy.config.connection_collapsing.revaliate_window_period”. Is this correct?
Thanks in advance,
Pete Walsh
Software Engineer
206-664-4150
Serving Stale Cached Content when origin server unreachable
Posted by "Walsh, Peter" <Pe...@disney.com>.
Hello all,
I found information in the Admin docs that indicate if the origin server doesn’t respond to a revalidation query, Traffic Server serves stale data along with a warning.
My question is: How often will Traffic Server try to revalidate the object when it has already been unable perform a revalidation query for that object before? My assumption is that Traffic Server will attempt to perform the revalidation on a single client request at a time and any other requests that come in for that same object will be served the stale cached object for as long as the time set in “proxy.config.connection_collapsing.revaliate_window_period”. Is this correct?
Thanks in advance,
Pete Walsh
Software Engineer
206-664-4150
Re: Proxying to dynamically changing origin servers
Posted by "ming.zym@gmail.com" <mi...@gmail.com>.
hmm, that is hard to solve by TS, why not setup some vpn tunnel, ie
using tap or tun etc, that will make things much easy for manage
在 2011-08-31三的 10:56 +0200，Rene Mayrhofer写道：
> Thanks for the answers! Based on these recommendations, we will try the DynDNS approach first. However, ....
>
> On Wednesday 31 August 2011 09:33:29 ming.zym@gmail.com wrote:
> > in this case, I think the ddns still be a very good solution for you,
> > what you need is:
> > 1, setup a ddns, in the intranet, ie a dhcp zone.
> > 2, setup the target origin server to update dns using ddns update.
> > 3, make sure the proxy server can get your ddns target resoled.
> > 4, config TS's hostdb following dns's TTL
> >
> > then you don't need to find someway to do magic mapping, all you need is
> > a ddns update with dhcp working.
>
> There is one use-case not previously mentioned in my email: some of the target origin servers may (as a future extension) not be reachable from the proxy, but may need to connect to the proxy server and keep the TCP connection alive from their end (firewall and NAT issues). That is something that is not even in detailled planning stage yet, but I'd like to keep the architecture open for this extension. Would Traffic Server allow us to write a network plugin that would re-use existing TCP sockets to origin servers instead of establishing a new one for each request, or would that go against the basic design?
>
> best regards,
> Rene
>
> > 在 2011-08-30二的 14:33 +0000，Igor Galić写道：
> > >
> > > ----- Original Message -----
> > > > Hi everybody,
> > > >
> > > > [Please CC me in replies, I am not currently subscribed to this
> > > > mailing
> > > > list.]
> > > >
> > > > For a research project, we currently have an interesting problem to
> > > > solve: to connect HTTPS clients to dynamically changing, internal
> > > > HTTPS
> > > > servers. One approach that we are currently evaluating is to use a
> > > > publicly accessible HTTP proxy with CONNECT support to "tunnel" the
> > > > HTTPS connections to the internal servers. However, the internal
> > > > addresses may change dynamically. The question is therefore if
> > > > Traffic
> > > > Server can be configured to (or if it is easy to write a plug-in to):
> > > >
> > > > a) be used in "normal" proxy server mode for clients with explicit
> > > > proxy
> > > > server configuration to use the CONNECT call for some HTTPS URLs; and
> > > >
> > > > b) for the origin server resolving to be done dynamically based on
> > > > internal look-up tables. E.g. the URL
> > > > https://my-example.local.com/whatever specified by any client in the
> > > > CONNECT request should be mapped to host 10.20.30.40 (the HTTPS
> > > > server
> > > > may use my-example.local.com as its server address, but the IP will
> > > > change dynamically).
> > >
> > > It seems to me all that is required is to set the DNS TTL very low.
> > >
> > >
> > > > I am aware that this is a mix between reverse proxy functionality
> > > > (mapping to internal servers) and normal proxying (CONNECT to
> > > > client-specified, different URLs). Based on the SDK documentation, I
> > > > am
> > > > also unsure which kind of plug-in would be required to make this
> > > > work.
> > > > The backup plan is to use a "normal" proxy with dynamic DNS for
> > > > resolving the internal IP addresses, but we would like to avoid this
> > > > complexity if possible.
> > > >
> > > > Are we on the right track and is this possible with Traffic Server?
> > > >
> > > > best regards,
> > > > Rene
> > >
> > > i
> > >
> >
> >
> >
>
Re: Proxying to dynamically changing origin servers
Posted by Rene Mayrhofer <re...@fh-hagenberg.at>.
Thanks for the answers! Based on these recommendations, we will try the DynDNS approach first. However, ....
On Wednesday 31 August 2011 09:33:29 ming.zym@gmail.com wrote:
> in this case, I think the ddns still be a very good solution for you,
> what you need is:
> 1, setup a ddns, in the intranet, ie a dhcp zone.
> 2, setup the target origin server to update dns using ddns update.
> 3, make sure the proxy server can get your ddns target resoled.
> 4, config TS's hostdb following dns's TTL
>
> then you don't need to find someway to do magic mapping, all you need is
> a ddns update with dhcp working.
There is one use-case not previously mentioned in my email: some of the target origin servers may (as a future extension) not be reachable from the proxy, but may need to connect to the proxy server and keep the TCP connection alive from their end (firewall and NAT issues). That is something that is not even in detailled planning stage yet, but I'd like to keep the architecture open for this extension. Would Traffic Server allow us to write a network plugin that would re-use existing TCP sockets to origin servers instead of establishing a new one for each request, or would that go against the basic design?
best regards,
Rene
> 在 2011-08-30二的 14:33 +0000，Igor Galić写道：
> >
> > ----- Original Message -----
> > > Hi everybody,
> > >
> > > [Please CC me in replies, I am not currently subscribed to this
> > > mailing
> > > list.]
> > >
> > > For a research project, we currently have an interesting problem to
> > > solve: to connect HTTPS clients to dynamically changing, internal
> > > HTTPS
> > > servers. One approach that we are currently evaluating is to use a
> > > publicly accessible HTTP proxy with CONNECT support to "tunnel" the
> > > HTTPS connections to the internal servers. However, the internal
> > > addresses may change dynamically. The question is therefore if
> > > Traffic
> > > Server can be configured to (or if it is easy to write a plug-in to):
> > >
> > > a) be used in "normal" proxy server mode for clients with explicit
> > > proxy
> > > server configuration to use the CONNECT call for some HTTPS URLs; and
> > >
> > > b) for the origin server resolving to be done dynamically based on
> > > internal look-up tables. E.g. the URL
> > > https://my-example.local.com/whatever specified by any client in the
> > > CONNECT request should be mapped to host 10.20.30.40 (the HTTPS
> > > server
> > > may use my-example.local.com as its server address, but the IP will
> > > change dynamically).
> >
> > It seems to me all that is required is to set the DNS TTL very low.
> >
> >
> > > I am aware that this is a mix between reverse proxy functionality
> > > (mapping to internal servers) and normal proxying (CONNECT to
> > > client-specified, different URLs). Based on the SDK documentation, I
> > > am
> > > also unsure which kind of plug-in would be required to make this
> > > work.
> > > The backup plan is to use a "normal" proxy with dynamic DNS for
> > > resolving the internal IP addresses, but we would like to avoid this
> > > complexity if possible.
> > >
> > > Are we on the right track and is this possible with Traffic Server?
> > >
> > > best regards,
> > > Rene
> >
> > i
> >
>
>
>
--
-----------------------------------------------------------------------------
Prof. (FH) Priv.-Doz. Dr. René Mayrhofer
Professor for Mobile Computing
School of Informatics/Communications/Media
University of Applied Sciences Upper Austria
FH OÖ Studienbetriebs GmbH
Softwarepark 11
4232 Hagenberg/Austria
Phone: +43 (0)7236 3888-2823
Fax: +43 (0)7236 3888-2899
E-Mail: rene.mayrhofer@fh-hagenberg.at
Web: www.fh-ooe.at
Firmenbuchgericht/Court of registry: Landesgericht Wels
Firmenbuchnummer/Company registration: FN 236729 g
Re: Proxying to dynamically changing origin servers
Posted by "ming.zym@gmail.com" <mi...@gmail.com>.
in this case, I think the ddns still be a very good solution for you,
what you need is:
1, setup a ddns, in the intranet, ie a dhcp zone.
2, setup the target origin server to update dns using ddns update.
3, make sure the proxy server can get your ddns target resoled.
4, config TS's hostdb following dns's TTL
then you don't need to find someway to do magic mapping, all you need is
a ddns update with dhcp working.
FYI
在 2011-08-30二的 14:33 +0000，Igor Galić写道：
>
> ----- Original Message -----
> > Hi everybody,
> >
> > [Please CC me in replies, I am not currently subscribed to this
> > mailing
> > list.]
> >
> > For a research project, we currently have an interesting problem to
> > solve: to connect HTTPS clients to dynamically changing, internal
> > HTTPS
> > servers. One approach that we are currently evaluating is to use a
> > publicly accessible HTTP proxy with CONNECT support to "tunnel" the
> > HTTPS connections to the internal servers. However, the internal
> > addresses may change dynamically. The question is therefore if
> > Traffic
> > Server can be configured to (or if it is easy to write a plug-in to):
> >
> > a) be used in "normal" proxy server mode for clients with explicit
> > proxy
> > server configuration to use the CONNECT call for some HTTPS URLs; and
> >
> > b) for the origin server resolving to be done dynamically based on
> > internal look-up tables. E.g. the URL
> > https://my-example.local.com/whatever specified by any client in the
> > CONNECT request should be mapped to host 10.20.30.40 (the HTTPS
> > server
> > may use my-example.local.com as its server address, but the IP will
> > change dynamically).
>
> It seems to me all that is required is to set the DNS TTL very low.
>
>
> > I am aware that this is a mix between reverse proxy functionality
> > (mapping to internal servers) and normal proxying (CONNECT to
> > client-specified, different URLs). Based on the SDK documentation, I
> > am
> > also unsure which kind of plug-in would be required to make this
> > work.
> > The backup plan is to use a "normal" proxy with dynamic DNS for
> > resolving the internal IP addresses, but we would like to avoid this
> > complexity if possible.
> >
> > Are we on the right track and is this possible with Traffic Server?
> >
> > best regards,
> > Rene
>
> i
>
Re: Proxying to dynamically changing origin servers
Posted by Igor Galić <i....@brainsware.org>.
----- Original Message -----
> Hi everybody,
>
> [Please CC me in replies, I am not currently subscribed to this
> mailing
> list.]
>
> For a research project, we currently have an interesting problem to
> solve: to connect HTTPS clients to dynamically changing, internal
> HTTPS
> servers. One approach that we are currently evaluating is to use a
> publicly accessible HTTP proxy with CONNECT support to "tunnel" the
> HTTPS connections to the internal servers. However, the internal
> addresses may change dynamically. The question is therefore if
> Traffic
> Server can be configured to (or if it is easy to write a plug-in to):
>
> a) be used in "normal" proxy server mode for clients with explicit
> proxy
> server configuration to use the CONNECT call for some HTTPS URLs; and
>
> b) for the origin server resolving to be done dynamically based on
> internal look-up tables. E.g. the URL
> https://my-example.local.com/whatever specified by any client in the
> CONNECT request should be mapped to host 10.20.30.40 (the HTTPS
> server
> may use my-example.local.com as its server address, but the IP will
> change dynamically).
It seems to me all that is required is to set the DNS TTL very low.
> I am aware that this is a mix between reverse proxy functionality
> (mapping to internal servers) and normal proxying (CONNECT to
> client-specified, different URLs). Based on the SDK documentation, I
> am
> also unsure which kind of plug-in would be required to make this
> work.
> The backup plan is to use a "normal" proxy with dynamic DNS for
> resolving the internal IP addresses, but we would like to avoid this
> complexity if possible.
>
> Are we on the right track and is this possible with Traffic Server?
>
> best regards,
> Rene
i
--
Igor Galić
Tel: +43 (0) 664 886 22 883
Mail: i.galic@brainsware.org
URL: http://brainsware.org/
GPG: 571B 8B8A FC97 266D BDA3 EF6F 43AD 80A4 5779 3257