You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Victor Danilchenko <vd...@cimpress.com> on 2016/01/20 19:22:49 UTC
Need help with a particular usage scenario
Hi all,
I need to set up a caching proxy to not work as a proxy, but rather to simply provide caching upon direct access. For example:
There's a Mediawiki installation running at http://mediawiki.mycompany.com
And I need to set up a proxy on an alternative port, http://mediawiki.mycompany.com:8080, which would do nothing but serve up the traffic from the default port, but cached (we aren't trying to cache the entire MediaWiki installation, but rather provide an alternative entry point for a specific use case).
I am having trouble figuring out what it would take to set up TrafficServer to do this. I set it up (Ubuntu Vivid, TS 3.2.4), but TS logs empty URLs (http:///<http://>) and of course fails to find them - presumably because it doesn't receive the customary proxy HTTP headers; so I hit http://mediawiki.mycompany.com:8080, and only get back 404s.
Is there an easy way to accomplish what I am looking to do? Here's the config I have right now.
In records.config (in addition to all the standard defaults):
CONFIG proxy.config.http.server_ports STRING 8080
CONFIG proxy.config.http.cache.http INT 1
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.url_remap.remap_required INT 1
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
And in remap.config:
reverse_map http://mediawiki.mycompany.com/ http://mediawiki.mycompany.com:8080/
I have never set up a proxy before.
Any suggestions on what I need to do? Thanks.
RE: Need help with a particular usage scenario
Posted by Victor Danilchenko <vd...@cimpress.com>.
So I kinda figured out what the problem is.
Basically, it’s specifically the host I was setting up TrafficServer on – it’s a personal VM, and I Imagine something was weird with DNS or routing there. I set up the same configuration on another machine, and it worked like a charm. I didn’t bother trying to figure out what’s wrong with my personal VM.
Many thanks for your help, Alan!
From: Alan Carroll [mailto:solidwallofcode@yahoo-inc.com]
Sent: Friday, January 22, 2016 10:51 AM
To: Victor Danilchenko <vd...@cimpress.com>; users@trafficserver.apache.org
Subject: Re: Need help with a particular usage scenario
The synthetic.txt requests are from traffic_cop for heartbeat purposes, to verify traffic_server is operational.
Looking at your log excerpt, it seems to be in the middle of the transaction. I see a failed connect but then an apparently successful retry (at the TCP layer, indicated by the NET_EVENT_OPEN and the creation of an http_ss session) but then the excerpt ends so I can't see how that turns out. Based on this, though, I'd guess the HTTP response is failing to parse.
It might be that, as a proxy, the request format is not what the server expects, particularly with regard to how the host value is present in the URL in the GET. You might try curl directly to the web server but with the --proxy option and see what happens. From the log you should also be able to see exactly what is sent by ATS to the origin server.
On Thursday, January 21, 2016 2:07 PM, Victor Danilchenko <vd...@cimpress.com>> wrote:
OK, I dropped the port 80 config, and enabled the debugging.
I then did some random poking, and I realized that requests from localhost succeed (i.e. when I use ‘curl’ on the server, or run Chromium) but fail from any other host. So I tried replacing all the hostnames in the remap file with IPs, but that didn’t help – the remote requests still time out.
I am not running AppArmor, and my /etc/hosts only contains the default entries (localhost definitions and somesuch). The DNS resolution on command line works in both directions (hostname to IP, and IP to hostname).
Anyway, the debug dump from a single request is huge, nearly 3000 lines. As best as I can figure, the following things might be relevant:
• Multiple failing requests for http://127.0.0.1:8083/synthetic.txt<http://127.0.0.1:8083/synthetic.txt> -- I have no idea where these are coming from.
• The request for http://vdanilchenko-corewiki.vistaprint.net:8080/phpinfo2.php<http://vdanilchenko-corewiki.vistaprint.net:8080/phpinfo2.php> gets received, processed, and forwarded to the origin server
• The response from the origin server result in the following log entriess (cleaned up for readability):
+++++++++ Incoming O.S. Response +++++++++
-- State Machine Id: 0
DEBUG: (http_seq) [HttpTransact::HandleResponse] Response not valid
DEBUG: (http_trans) [handle_response_from_server] (hrfs)
DEBUG: (http_trans) [1] failed to connect [5] to 10.87.40.225
DEBUG: (http_trans) [retry_server_connection_not_open] attempts now: 2, max: 3
DEBUG: (http_trans) [handle_response_from_server] Error. Retrying...
DEBUG: (http) [0] State Transition: ORIGIN_SERVER_OPEN -> ORIGIN_SERVER_OPEN
DEBUG: (http_track) entered inside do_http_server_open
DEBUG: (http) [0] open connection to vdanilchenko-corewiki.vistaprint.net:<http://vdanilchenko-corewiki.vistaprint.net:> 10.87.40.225:80<http://10.87.40.225:80>
DEBUG: (http_seq) [HttpSM::do_http_server_open] Sending request to server
DEBUG: (http) calling netProcessor.connect_re
DEBUG: (http) [0] [HttpSM::main_handler, NET_EVENT_OPEN]
DEBUG: (http_track) entered inside state_http_server_open
DEBUG: (http) [0] [&HttpSM::state_http_server_open, NET_EVENT_OPEN]
DEBUG: (http_ss) [4] session born, netvc 0x2b59ec0150e0
Each such set of log entries is followed by a number of apparently failing requests for http://127.0.0.1:8083/synthetic.txt<http://127.0.0.1:8083/synthetic.txt> – even though I can access that URL and retrieve the ‘synthetics.txt’ file (it’s 60 lines of ‘abcdefghijklmnopqrstuvwxyz’).
Manually retrieving http://10.87.40.225/phpinfo2.php<http://10.87.40.225/phpinfo2.php> (on non-proxy port) works like a charm, though, so I still have no idea why TrafficServer is failing to connect.
I tried monitoring traffic on port 80 via ngrep, and when I do regular request to origin server (on port 80), it works fine – I observe the request and response as expected; but when I make a request to the proxy (on port 8080), there’s simply nothing going across the port 80. Ngrep is just sitting there and printing packet hashmarks, but no content.
So it looks like trafficserver is not merely not getting a response, it’s never actually making the request… as long as the client is remote. If the client is local, everything works as expected.
From: Alan Carroll [mailto:solidwallofcode@yahoo-inc.com]
Sent: Wednesday, January 20, 2016 3:19 PM
To: users@trafficserver.apache.org<ma...@trafficserver.apache.org>
Subject: Re: Need help with a particular usage scenario
A couple of things to try -
1) Drop the :80
2) Turn on debug tags with a value of "http|dns" and see what you get. Those tend to be informative in this regard, as you will be able to see exactly what ATS thinks it is connecting to.
On Wednesday, January 20, 2016 2:05 PM, Victor Danilchenko <vd...@cimpress.com>> wrote:
Ah, thank you very much, updating the maps did change things! Right now I am getting error messages that kinda make sense.
So, here’s my new remap settings:
map http://mediawiki.mycompany.com:8080/<http://mediawiki.mycompany.com:8080/> http://localhost:80/<http://localhost/>
reverse_map http://localhost:80/<http://localhost/> http://mediawiki.mycompany.com:8080/<http://mediawiki.mycompany.com:8080/>
When I try to hit http://mediawiki.mycompany.com:8080/phpinfo2.php<http://mediawiki.mycompany.com:8080/phpinfo2.php> in the browser now, I get a 502 response after a lengthy timeout:
internal error - server connection terminated
and on the server, in error.log, the following gets recorded:
20160120.14h51m15s RESPONSE: sent 10.87.40.108 status 502 (Connect Error <internal error - server connection terminated/-19999>) for 'http://localhost:80/phpinfo2.php'
I tried this both using localhost in the maps, and FQDNs everywhere, the result is the same. I most definitely can get this URL on the server using curl:
$ curl -Is http://localhost:80/phpinfo2.php<http://localhost/phpinfo2.php> | grep ^HTTP
HTTP/1.1 200 OK
So it looks like for some weird reason, TrafficServer is trying – and failing – to contact the origin provider.
Any idea what might be going wrong?
Thanks so much for your help!
From: Alan Carroll [mailto:solidwallofcode@yahoo-inc.com]
Sent: Wednesday, January 20, 2016 2:09 PM
To: users@trafficserver.apache.org<ma...@trafficserver.apache.org>
Subject: Re: Need help with a particular usage scenario
Hmmm. A parent proxy set up might work better. The logging of empty URLs is a bug in the logging code which has been fixed (current ATS version is 6.0). If you are doing remap, you need a forward map as well like
map http://mediawiki.mycompany.com:8080/<http://mediawiki.mycompany.com:8080/> http://mediawiki.mycompany/<http://mediawiki.mycompany/>
reverse_map, if I recall correctly, only applies to response headers from the origin, to modify them to use the proxy URL instead of the origin URL (as supplied by the origin).
I'd recommend moving up in versions, to at least 5.3.x.
On Wednesday, January 20, 2016 12:23 PM, Victor Danilchenko <vd...@cimpress.com>> wrote:
Hi all,
I need to set up a caching proxy to not work as a proxy, but rather to simply provide caching upon direct access. For example:
There’s a Mediawiki installation running at http://mediawiki.mycompany.com<http://mediawiki.mycompany.com/>
And I need to set up a proxy on an alternative port, http://mediawiki.mycompany.com:8080<http://mediawiki.mycompany.com:8080/>, which would do nothing but serve up the traffic from the default port, but cached (we aren’t trying to cache the entire MediaWiki installation, but rather provide an alternative entry point for a specific use case).
I am having trouble figuring out what it would take to set up TrafficServer to do this. I set it up (Ubuntu Vivid, TS 3.2.4), but TS logs empty URLs (http:///) and of course fails to find them – presumably because it doesn’t receive the customary proxy HTTP headers; so I hit http://mediawiki.mycompany.com:8080<http://mediawiki.mycompany.com:8080/>, and only get back 404s.
Is there an easy way to accomplish what I am looking to do? Here’s the config I have right now.
In records.config (in addition to all the standard defaults):
CONFIG proxy.config.http.server_ports STRING 8080
CONFIG proxy.config.http.cache.http INT 1
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.url_remap.remap_required INT 1
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
And in remap.config:
reverse_map http://mediawiki.mycompany.com/<http://mediawiki.mycompany.com/> http://mediawiki.mycompany.com:8080/<http://mediawiki.mycompany.com:8080/>
I have never set up a proxy before.
Any suggestions on what I need to do? Thanks.
Re: Need help with a particular usage scenario
Posted by Alan Carroll <so...@yahoo-inc.com>.
The synthetic.txt requests are from traffic_cop for heartbeat purposes, to verify traffic_server is operational.
Looking at your log excerpt, it seems to be in the middle of the transaction. I see a failed connect but then an apparently successful retry (at the TCP layer, indicated by the NET_EVENT_OPEN and the creation of an http_ss session) but then the excerpt ends so I can't see how that turns out. Based on this, though, I'd guess the HTTP response is failing to parse.
It might be that, as a proxy, the request format is not what the server expects, particularly with regard to how the host value is present in the URL in the GET. You might try curl directly to the web server but with the --proxy option and see what happens. From the log you should also be able to see exactly what is sent by ATS to the origin server.
On Thursday, January 21, 2016 2:07 PM, Victor Danilchenko <vd...@cimpress.com> wrote:
#yiv0619496581 #yiv0619496581 -- _filtered #yiv0619496581 {font-family:Wingdings;panose-1:5 0 0 0 0 0 0 0 0 0;} _filtered #yiv0619496581 {panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv0619496581 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv0619496581 {panose-1:2 11 6 9 4 5 4 2 2 4;} _filtered #yiv0619496581 {panose-1:0 0 0 0 0 0 0 0 0 0;}#yiv0619496581 #yiv0619496581 p.yiv0619496581MsoNormal, #yiv0619496581 li.yiv0619496581MsoNormal, #yiv0619496581 div.yiv0619496581MsoNormal {margin:0in;margin-bottom:.0001pt;font-size:12.0pt;}#yiv0619496581 a:link, #yiv0619496581 span.yiv0619496581MsoHyperlink {color:blue;text-decoration:underline;}#yiv0619496581 a:visited, #yiv0619496581 span.yiv0619496581MsoHyperlinkFollowed {color:purple;text-decoration:underline;}#yiv0619496581 p.yiv0619496581MsoListParagraph, #yiv0619496581 li.yiv0619496581MsoListParagraph, #yiv0619496581 div.yiv0619496581MsoListParagraph {margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;font-size:12.0pt;}#yiv0619496581 p.yiv0619496581msonormal, #yiv0619496581 li.yiv0619496581msonormal, #yiv0619496581 div.yiv0619496581msonormal {margin-right:0in;margin-left:0in;font-size:12.0pt;}#yiv0619496581 p.yiv0619496581msochpdefault, #yiv0619496581 li.yiv0619496581msochpdefault, #yiv0619496581 div.yiv0619496581msochpdefault {margin-right:0in;margin-left:0in;font-size:12.0pt;}#yiv0619496581 p.yiv0619496581msonormal1, #yiv0619496581 li.yiv0619496581msonormal1, #yiv0619496581 div.yiv0619496581msonormal1 {margin-right:0in;margin-left:0in;font-size:12.0pt;}#yiv0619496581 p.yiv0619496581msochpdefault1, #yiv0619496581 li.yiv0619496581msochpdefault1, #yiv0619496581 div.yiv0619496581msochpdefault1 {margin-right:0in;margin-left:0in;font-size:12.0pt;}#yiv0619496581 p.yiv0619496581msonormal2, #yiv0619496581 li.yiv0619496581msonormal2, #yiv0619496581 div.yiv0619496581msonormal2 {margin:0in;margin-bottom:.0001pt;font-size:12.0pt;}#yiv0619496581 p.yiv0619496581msonormal3, #yiv0619496581 li.yiv0619496581msonormal3, #yiv0619496581 div.yiv0619496581msonormal3 {margin-right:0in;margin-left:0in;font-size:12.0pt;}#yiv0619496581 p.yiv0619496581msochpdefault2, #yiv0619496581 li.yiv0619496581msochpdefault2, #yiv0619496581 div.yiv0619496581msochpdefault2 {margin-right:0in;margin-left:0in;font-size:12.0pt;}#yiv0619496581 p.yiv0619496581msonormal11, #yiv0619496581 li.yiv0619496581msonormal11, #yiv0619496581 div.yiv0619496581msonormal11 {margin:0in;margin-bottom:.0001pt;font-size:11.0pt;}#yiv0619496581 p.yiv0619496581msochpdefault11, #yiv0619496581 li.yiv0619496581msochpdefault11, #yiv0619496581 div.yiv0619496581msochpdefault11 {margin-right:0in;margin-left:0in;font-size:12.0pt;}#yiv0619496581 span.yiv0619496581msohyperlink {}#yiv0619496581 span.yiv0619496581msohyperlinkfollowed {}#yiv0619496581 span.yiv0619496581msohyperlink1 {}#yiv0619496581 span.yiv0619496581msohyperlinkfollowed1 {}#yiv0619496581 span.yiv0619496581emailstyle171 {}#yiv0619496581 span.yiv0619496581emailstyle27 {}#yiv0619496581 span.yiv0619496581msohyperlink2 {color:blue;text-decoration:underline;}#yiv0619496581 span.yiv0619496581msohyperlinkfollowed2 {color:purple;text-decoration:underline;}#yiv0619496581 span.yiv0619496581msohyperlink11 {color:#0563C1;text-decoration:underline;}#yiv0619496581 span.yiv0619496581msohyperlinkfollowed11 {color:#954F72;text-decoration:underline;}#yiv0619496581 span.yiv0619496581emailstyle1711 {color:windowtext;}#yiv0619496581 span.yiv0619496581emailstyle271 {color:#1F497D;}#yiv0619496581 span.yiv0619496581EmailStyle39 {color:#1F497D;}#yiv0619496581 .yiv0619496581MsoChpDefault {font-size:10.0pt;} _filtered #yiv0619496581 {margin:1.0in 1.0in 1.0in 1.0in;}#yiv0619496581 div.yiv0619496581WordSection1 {}#yiv0619496581 _filtered #yiv0619496581 {} _filtered #yiv0619496581 {font-family:Symbol;} _filtered #yiv0619496581 {} _filtered #yiv0619496581 {font-family:Wingdings;} _filtered #yiv0619496581 {font-family:Symbol;} _filtered #yiv0619496581 {} _filtered #yiv0619496581 {font-family:Wingdings;} _filtered #yiv0619496581 {font-family:Symbol;} _filtered #yiv0619496581 {} _filtered #yiv0619496581 {font-family:Wingdings;}#yiv0619496581 ol {margin-bottom:0in;}#yiv0619496581 ul {margin-bottom:0in;}#yiv0619496581 OK, I dropped the port 80 config, and enabled the debugging. I then did some random poking, and I realized that requests fromlocalhost succeed (i.e. when I use ‘curl’ on the server, or run Chromium) but fail from any other host. So I tried replacing all the hostnames in the remap file with IPs, but that didn’t help – the remote requests still time out. I am not running AppArmor, and my /etc/hosts only contains the default entries (localhost definitions and somesuch). The DNS resolution on command line works in both directions (hostname to IP, and IP to hostname). Anyway, the debug dump from a single request is huge, nearly 3000 lines. As best as I can figure, the following things might be relevant: · Multiple failing requests for http://127.0.0.1:8083/synthetic.txt -- I have no idea where these are coming from. · The request forhttp://vdanilchenko-corewiki.vistaprint.net:8080/phpinfo2.php gets received, processed, and forwarded to the origin server · The response from the origin server result in the following log entriess (cleaned up for readability):
+++++++++ Incoming O.S. Response +++++++++ -- State Machine Id: 0 DEBUG: (http_seq) [HttpTransact::HandleResponse]Response not valid DEBUG: (http_trans) [handle_response_from_server] (hrfs) DEBUG: (http_trans) [1]failed to connect [5] to 10.87.40.225 DEBUG: (http_trans) [retry_server_connection_not_open] attempts now: 2, max: 3 DEBUG: (http_trans) [handle_response_from_server] Error. Retrying... DEBUG: (http) [0] State Transition: ORIGIN_SERVER_OPEN -> ORIGIN_SERVER_OPEN DEBUG: (http_track) entered inside do_http_server_open DEBUG: (http) [0] open connection to vdanilchenko-corewiki.vistaprint.net: 10.87.40.225:80 DEBUG: (http_seq) [HttpSM::do_http_server_open] Sending request to server DEBUG: (http) calling netProcessor.connect_re DEBUG: (http) [0] [HttpSM::main_handler, NET_EVENT_OPEN] DEBUG: (http_track) entered inside state_http_server_open DEBUG: (http) [0] [&HttpSM::state_http_server_open, NET_EVENT_OPEN] DEBUG: (http_ss) [4] session born, netvc 0x2b59ec0150e0 Each such set of log entries is followed by a number of apparently failing requests forhttp://127.0.0.1:8083/synthetic.txt – even though I can access that URL and retrieve the ‘synthetics.txt’ file (it’s 60 lines of ‘abcdefghijklmnopqrstuvwxyz’). Manually retrievinghttp://10.87.40.225/phpinfo2.php (on non-proxy port) works like a charm, though, so I still have no idea why TrafficServer is failing to connect. I tried monitoring traffic on port 80 via ngrep, and when I do regular request to origin server (on port 80), it works fine – I observe the request and response as expected; but when I make a request to the proxy (on port 8080), there’s simply nothing going across the port 80. Ngrep is just sitting there and printing packet hashmarks, but no content. So it looks like trafficserver is not merely not getting a response, it’s never actually making the request… as long as the client is remote. If the client is local, everything works as expected. From: Alan Carroll [mailto:solidwallofcode@yahoo-inc.com]
Sent: Wednesday, January 20, 2016 3:19 PM
To: users@trafficserver.apache.org
Subject: Re: Need help with a particular usage scenario A couple of things to try - 1) Drop the :80 2) Turn on debug tags with a value of "http|dns" and see what you get. Those tend to be informative in this regard, as you will be able to see exactly what ATS thinks it is connecting to. On Wednesday, January 20, 2016 2:05 PM, Victor Danilchenko <vd...@cimpress.com> wrote: Ah, thank you very much, updating the maps did change things! Right now I am getting error messages that kinda make sense. So, here’s my new remap settings: maphttp://mediawiki.mycompany.com:8080/ http://localhost:80/ reverse_maphttp://localhost:80/ http://mediawiki.mycompany.com:8080/ When I try to hithttp://mediawiki.mycompany.com:8080/phpinfo2.php in the browser now, I get a 502 response after a lengthy timeout: internal error - server connection terminated and on the server, in error.log, the following gets recorded: 20160120.14h51m15s RESPONSE: sent 10.87.40.108 status 502 (Connect Error <internal error - server connection terminated/-19999>) for 'http://localhost:80/phpinfo2.php' I tried this both using localhost in the maps,and FQDNs everywhere, the result is the same. I most definitely can get this URL on the server using curl: $ curl -Ishttp://localhost:80/phpinfo2.php| grep ^HTTP HTTP/1.1 200 OK So it looks like for some weird reason, TrafficServer is trying – and failing – to contact the origin provider. Any idea what might be going wrong? Thanks so much for your help! From: Alan Carroll [mailto:solidwallofcode@yahoo-inc.com]
Sent: Wednesday, January 20, 2016 2:09 PM
To: users@trafficserver.apache.org
Subject: Re: Need help with a particular usage scenario Hmmm. A parent proxy set up might work better. The logging of empty URLs is a bug in the logging code which has been fixed (current ATS version is 6.0). If you are doing remap, you need a forward map as well like maphttp://mediawiki.mycompany.com:8080/http://mediawiki.mycompany/ reverse_map, if I recall correctly, only applies to response headers from the origin, to modify them to use the proxy URL instead of the origin URL (as supplied by the origin). I'd recommend moving up in versions, to at least 5.3.x. On Wednesday, January 20, 2016 12:23 PM, Victor Danilchenko <vd...@cimpress.com> wrote: Hi all, I need to set up a caching proxy to not work as a proxy, but rather to simply provide caching upon direct access. For example: There’s a Mediawiki installation running athttp://mediawiki.mycompany.com And I need to set up a proxy on an alternative port,http://mediawiki.mycompany.com:8080, which would do nothing but serve up the traffic from the default port, but cached (we aren’t trying to cache the entire MediaWiki installation, but rather provide an alternative entry point for a specific use case). I am having trouble figuring out what it would take to set up TrafficServer to do this. I set it up (Ubuntu Vivid, TS 3.2.4), but TS logs empty URLs (http:///) and of course fails to find them – presumably because it doesn’t receive the customary proxy HTTP headers; so I hithttp://mediawiki.mycompany.com:8080, and only get back 404s. Is there an easy way to accomplish what I am looking to do? Here’s the config I have right now. In records.config (in addition to all the standard defaults): CONFIG proxy.config.http.server_ports STRING 8080 CONFIG proxy.config.http.cache.http INT 1 CONFIG proxy.config.reverse_proxy.enabled INT 1 CONFIG proxy.config.url_remap.remap_required INT 1 CONFIG proxy.config.url_remap.pristine_host_hdr INT 1 And in remap.config: reverse_maphttp://mediawiki.mycompany.com/ http://mediawiki.mycompany.com:8080/ I have never set up a proxy before. Any suggestions on what I need to do? Thanks.
RE: Need help with a particular usage scenario
Posted by Victor Danilchenko <vd...@cimpress.com>.
OK, I dropped the port 80 config, and enabled the debugging.
I then did some random poking, and I realized that requests from localhost succeed (i.e. when I use ‘curl’ on the server, or run Chromium) but fail from any other host. So I tried replacing all the hostnames in the remap file with IPs, but that didn’t help – the remote requests still time out.
I am not running AppArmor, and my /etc/hosts only contains the default entries (localhost definitions and somesuch). The DNS resolution on command line works in both directions (hostname to IP, and IP to hostname).
Anyway, the debug dump from a single request is huge, nearly 3000 lines. As best as I can figure, the following things might be relevant:
· Multiple failing requests for http://127.0.0.1:8083/synthetic.txt -- I have no idea where these are coming from.
· The request for http://vdanilchenko-corewiki.vistaprint.net:8080/phpinfo2.php gets received, processed, and forwarded to the origin server
· The response from the origin server result in the following log entriess (cleaned up for readability):
+++++++++ Incoming O.S. Response +++++++++
-- State Machine Id: 0
DEBUG: (http_seq) [HttpTransact::HandleResponse] Response not valid
DEBUG: (http_trans) [handle_response_from_server] (hrfs)
DEBUG: (http_trans) [1] failed to connect [5] to 10.87.40.225
DEBUG: (http_trans) [retry_server_connection_not_open] attempts now: 2, max: 3
DEBUG: (http_trans) [handle_response_from_server] Error. Retrying...
DEBUG: (http) [0] State Transition: ORIGIN_SERVER_OPEN -> ORIGIN_SERVER_OPEN
DEBUG: (http_track) entered inside do_http_server_open
DEBUG: (http) [0] open connection to vdanilchenko-corewiki.vistaprint.net: 10.87.40.225:80
DEBUG: (http_seq) [HttpSM::do_http_server_open] Sending request to server
DEBUG: (http) calling netProcessor.connect_re
DEBUG: (http) [0] [HttpSM::main_handler, NET_EVENT_OPEN]
DEBUG: (http_track) entered inside state_http_server_open
DEBUG: (http) [0] [&HttpSM::state_http_server_open, NET_EVENT_OPEN]
DEBUG: (http_ss) [4] session born, netvc 0x2b59ec0150e0
Each such set of log entries is followed by a number of apparently failing requests for http://127.0.0.1:8083/synthetic.txt – even though I can access that URL and retrieve the ‘synthetics.txt’ file (it’s 60 lines of ‘abcdefghijklmnopqrstuvwxyz’).
Manually retrieving http://10.87.40.225/phpinfo2.php (on non-proxy port) works like a charm, though, so I still have no idea why TrafficServer is failing to connect.
I tried monitoring traffic on port 80 via ngrep, and when I do regular request to origin server (on port 80), it works fine – I observe the request and response as expected; but when I make a request to the proxy (on port 8080), there’s simply nothing going across the port 80. Ngrep is just sitting there and printing packet hashmarks, but no content.
So it looks like trafficserver is not merely not getting a response, it’s never actually making the request… as long as the client is remote. If the client is local, everything works as expected.
From: Alan Carroll [mailto:solidwallofcode@yahoo-inc.com]
Sent: Wednesday, January 20, 2016 3:19 PM
To: users@trafficserver.apache.org
Subject: Re: Need help with a particular usage scenario
A couple of things to try -
1) Drop the :80
2) Turn on debug tags with a value of "http|dns" and see what you get. Those tend to be informative in this regard, as you will be able to see exactly what ATS thinks it is connecting to.
On Wednesday, January 20, 2016 2:05 PM, Victor Danilchenko <vd...@cimpress.com>> wrote:
Ah, thank you very much, updating the maps did change things! Right now I am getting error messages that kinda make sense.
So, here’s my new remap settings:
map http://mediawiki.mycompany.com:8080/<http://mediawiki.mycompany.com:8080/> http://localhost:80/
reverse_map http://localhost:80/ http://mediawiki.mycompany.com:8080/<http://mediawiki.mycompany.com:8080/>
When I try to hit http://mediawiki.mycompany.com:8080/phpinfo2.php<http://mediawiki.mycompany.com:8080/phpinfo2.php> in the browser now, I get a 502 response after a lengthy timeout:
internal error - server connection terminated
and on the server, in error.log, the following gets recorded:
20160120.14h51m15s RESPONSE: sent 10.87.40.108 status 502 (Connect Error <internal error - server connection terminated/-19999>) for 'http://localhost:80/phpinfo2.php'
I tried this both using localhost in the maps, and FQDNs everywhere, the result is the same. I most definitely can get this URL on the server using curl:
$ curl -Is http://localhost:80/phpinfo2.php | grep ^HTTP
HTTP/1.1 200 OK
So it looks like for some weird reason, TrafficServer is trying – and failing – to contact the origin provider.
Any idea what might be going wrong?
Thanks so much for your help!
From: Alan Carroll [mailto:solidwallofcode@yahoo-inc.com]
Sent: Wednesday, January 20, 2016 2:09 PM
To: users@trafficserver.apache.org<ma...@trafficserver.apache.org>
Subject: Re: Need help with a particular usage scenario
Hmmm. A parent proxy set up might work better. The logging of empty URLs is a bug in the logging code which has been fixed (current ATS version is 6.0). If you are doing remap, you need a forward map as well like
map http://mediawiki.mycompany.com:8080/<http://mediawiki.mycompany.com:8080/> http://mediawiki.mycompany/<http://mediawiki.mycompany/>
reverse_map, if I recall correctly, only applies to response headers from the origin, to modify them to use the proxy URL instead of the origin URL (as supplied by the origin).
I'd recommend moving up in versions, to at least 5.3.x.
On Wednesday, January 20, 2016 12:23 PM, Victor Danilchenko <vd...@cimpress.com>> wrote:
Hi all,
I need to set up a caching proxy to not work as a proxy, but rather to simply provide caching upon direct access. For example:
There’s a Mediawiki installation running at http://mediawiki.mycompany.com<http://mediawiki.mycompany.com/>
And I need to set up a proxy on an alternative port, http://mediawiki.mycompany.com:8080<http://mediawiki.mycompany.com:8080/>, which would do nothing but serve up the traffic from the default port, but cached (we aren’t trying to cache the entire MediaWiki installation, but rather provide an alternative entry point for a specific use case).
I am having trouble figuring out what it would take to set up TrafficServer to do this. I set it up (Ubuntu Vivid, TS 3.2.4), but TS logs empty URLs (http:///<http://>) and of course fails to find them – presumably because it doesn’t receive the customary proxy HTTP headers; so I hit http://mediawiki.mycompany.com:8080<http://mediawiki.mycompany.com:8080/>, and only get back 404s.
Is there an easy way to accomplish what I am looking to do? Here’s the config I have right now.
In records.config (in addition to all the standard defaults):
CONFIG proxy.config.http.server_ports STRING 8080
CONFIG proxy.config.http.cache.http INT 1
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.url_remap.remap_required INT 1
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
And in remap.config:
reverse_map http://mediawiki.mycompany.com/<http://mediawiki.mycompany.com/> http://mediawiki.mycompany.com:8080/<http://mediawiki.mycompany.com:8080/>
I have never set up a proxy before.
Any suggestions on what I need to do? Thanks.
Re: Need help with a particular usage scenario
Posted by Alan Carroll <so...@yahoo-inc.com>.
A couple of things to try -
1) Drop the :80
2) Turn on debug tags with a value of "http|dns" and see what you get. Those tend to be informative in this regard, as you will be able to see exactly what ATS thinks it is connecting to.
On Wednesday, January 20, 2016 2:05 PM, Victor Danilchenko <vd...@cimpress.com> wrote:
#yiv2744081655 #yiv2744081655 -- _filtered #yiv2744081655 {font-family:Helvetica;panose-1:2 11 6 4 2 2 2 2 2 4;} _filtered #yiv2744081655 {panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv2744081655 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv2744081655 {panose-1:2 11 6 9 4 5 4 2 2 4;}#yiv2744081655 #yiv2744081655 p.yiv2744081655MsoNormal, #yiv2744081655 li.yiv2744081655MsoNormal, #yiv2744081655 div.yiv2744081655MsoNormal {margin:0in;margin-bottom:.0001pt;font-size:12.0pt;}#yiv2744081655 a:link, #yiv2744081655 span.yiv2744081655MsoHyperlink {color:blue;text-decoration:underline;}#yiv2744081655 a:visited, #yiv2744081655 span.yiv2744081655MsoHyperlinkFollowed {color:purple;text-decoration:underline;}#yiv2744081655 p.yiv2744081655msonormal, #yiv2744081655 li.yiv2744081655msonormal, #yiv2744081655 div.yiv2744081655msonormal {margin-right:0in;margin-left:0in;font-size:12.0pt;}#yiv2744081655 p.yiv2744081655msochpdefault, #yiv2744081655 li.yiv2744081655msochpdefault, #yiv2744081655 div.yiv2744081655msochpdefault {margin-right:0in;margin-left:0in;font-size:12.0pt;}#yiv2744081655 span.yiv2744081655msohyperlink {}#yiv2744081655 span.yiv2744081655msohyperlinkfollowed {}#yiv2744081655 span.yiv2744081655emailstyle17 {}#yiv2744081655 p.yiv2744081655msonormal1, #yiv2744081655 li.yiv2744081655msonormal1, #yiv2744081655 div.yiv2744081655msonormal1 {margin:0in;margin-bottom:.0001pt;font-size:11.0pt;}#yiv2744081655 span.yiv2744081655msohyperlink1 {color:#0563C1;text-decoration:underline;}#yiv2744081655 span.yiv2744081655msohyperlinkfollowed1 {color:#954F72;text-decoration:underline;}#yiv2744081655 span.yiv2744081655emailstyle171 {color:windowtext;}#yiv2744081655 p.yiv2744081655msochpdefault1, #yiv2744081655 li.yiv2744081655msochpdefault1, #yiv2744081655 div.yiv2744081655msochpdefault1 {margin-right:0in;margin-left:0in;font-size:12.0pt;}#yiv2744081655 span.yiv2744081655EmailStyle27 {color:#1F497D;}#yiv2744081655 .yiv2744081655MsoChpDefault {font-size:10.0pt;} _filtered #yiv2744081655 {margin:1.0in 1.0in 1.0in 1.0in;}#yiv2744081655 div.yiv2744081655WordSection1 {}#yiv2744081655 Ah, thank you very much, updating the maps did change things! Right now I am getting error messages that kinda make sense. So, here’s my new remap settings: map http://mediawiki.mycompany.com:8080/ http://localhost:80/ reverse_map http://localhost:80/ http://mediawiki.mycompany.com:8080/ When I try to hithttp://mediawiki.mycompany.com:8080/phpinfo2.php in the browser now, I get a 502 response after a lengthy timeout: internal error - server connection terminated and on the server, in error.log, the following gets recorded: 20160120.14h51m15s RESPONSE: sent 10.87.40.108 status 502 (Connect Error <internal error - server connection terminated/-19999>) for 'http://localhost:80/phpinfo2.php' I tried this both using localhost in the maps, and FQDNs everywhere, the result is the same. I most definitely can get this URL on the server using curl: $ curl -Is http://localhost:80/phpinfo2.php | grep ^HTTP HTTP/1.1 200 OK So it looks like for some weird reason, TrafficServer is trying – and failing – to contact the origin provider. Any idea what might be going wrong? Thanks so much for your help! From: Alan Carroll [mailto:solidwallofcode@yahoo-inc.com]
Sent: Wednesday, January 20, 2016 2:09 PM
To: users@trafficserver.apache.org
Subject: Re: Need help with a particular usage scenario Hmmm. A parent proxy set up might work better. The logging of empty URLs is a bug in the logging code which has been fixed (current ATS version is 6.0). If you are doing remap, you need a forward map as well like maphttp://mediawiki.mycompany.com:8080/ http://mediawiki.mycompany/ reverse_map, if I recall correctly, only applies to response headers from the origin, to modify them to use the proxy URL instead of the origin URL (as supplied by the origin). I'd recommend moving up in versions, to at least 5.3.x. On Wednesday, January 20, 2016 12:23 PM, Victor Danilchenko <vd...@cimpress.com> wrote: Hi all, I need to set up a caching proxy to not work as a proxy, but rather to simply provide caching upon direct access. For example: There’s a Mediawiki installation running athttp://mediawiki.mycompany.com And I need to set up a proxy on an alternative port,http://mediawiki.mycompany.com:8080, which would do nothing but serve up the traffic from the default port, but cached (we aren’t trying to cache the entire MediaWiki installation, but rather provide an alternative entry point for a specific use case). I am having trouble figuring out what it would take to set up TrafficServer to do this. I set it up (Ubuntu Vivid, TS 3.2.4), but TS logs empty URLs (http:///) and of course fails to find them – presumably because it doesn’t receive the customary proxy HTTP headers; so I hithttp://mediawiki.mycompany.com:8080, and only get back 404s. Is there an easy way to accomplish what I am looking to do? Here’s the config I have right now. In records.config (in addition to all the standard defaults): CONFIG proxy.config.http.server_ports STRING 8080 CONFIG proxy.config.http.cache.http INT 1 CONFIG proxy.config.reverse_proxy.enabled INT 1 CONFIG proxy.config.url_remap.remap_required INT 1 CONFIG proxy.config.url_remap.pristine_host_hdr INT 1 And in remap.config: reverse_maphttp://mediawiki.mycompany.com/ http://mediawiki.mycompany.com:8080/ I have never set up a proxy before. Any suggestions on what I need to do? Thanks.
RE: Need help with a particular usage scenario
Posted by Victor Danilchenko <vd...@cimpress.com>.
Ah, thank you very much, updating the maps did change things! Right now I am getting error messages that kinda make sense.
So, here’s my new remap settings:
map http://mediawiki.mycompany.com:8080/ http://localhost:80/
reverse_map http://localhost:80/ http://mediawiki.mycompany.com:8080/
When I try to hit http://mediawiki.mycompany.com:8080/phpinfo2.php in the browser now, I get a 502 response after a lengthy timeout:
internal error - server connection terminated
and on the server, in error.log, the following gets recorded:
20160120.14h51m15s RESPONSE: sent 10.87.40.108 status 502 (Connect Error <internal error - server connection terminated/-19999>) for 'http://localhost:80/phpinfo2.php'
I tried this both using localhost in the maps, and FQDNs everywhere, the result is the same. I most definitely can get this URL on the server using curl:
$ curl -Is http://localhost:80/phpinfo2.php | grep ^HTTP
HTTP/1.1 200 OK
So it looks like for some weird reason, TrafficServer is trying – and failing – to contact the origin provider.
Any idea what might be going wrong?
Thanks so much for your help!
From: Alan Carroll [mailto:solidwallofcode@yahoo-inc.com]
Sent: Wednesday, January 20, 2016 2:09 PM
To: users@trafficserver.apache.org
Subject: Re: Need help with a particular usage scenario
Hmmm. A parent proxy set up might work better. The logging of empty URLs is a bug in the logging code which has been fixed (current ATS version is 6.0). If you are doing remap, you need a forward map as well like
map http://mediawiki.mycompany.com:8080/<http://mediawiki.mycompany.com:8080/> http://mediawiki.mycompany/<http://mediawiki.mycompany/>
reverse_map, if I recall correctly, only applies to response headers from the origin, to modify them to use the proxy URL instead of the origin URL (as supplied by the origin).
I'd recommend moving up in versions, to at least 5.3.x.
On Wednesday, January 20, 2016 12:23 PM, Victor Danilchenko <vd...@cimpress.com>> wrote:
Hi all,
I need to set up a caching proxy to not work as a proxy, but rather to simply provide caching upon direct access. For example:
There’s a Mediawiki installation running at http://mediawiki.mycompany.com<http://mediawiki.mycompany.com/>
And I need to set up a proxy on an alternative port, http://mediawiki.mycompany.com:8080<http://mediawiki.mycompany.com:8080/>, which would do nothing but serve up the traffic from the default port, but cached (we aren’t trying to cache the entire MediaWiki installation, but rather provide an alternative entry point for a specific use case).
I am having trouble figuring out what it would take to set up TrafficServer to do this. I set it up (Ubuntu Vivid, TS 3.2.4), but TS logs empty URLs (http:///) and of course fails to find them – presumably because it doesn’t receive the customary proxy HTTP headers; so I hit http://mediawiki.mycompany.com:8080<http://mediawiki.mycompany.com:8080/>, and only get back 404s.
Is there an easy way to accomplish what I am looking to do? Here’s the config I have right now.
In records.config (in addition to all the standard defaults):
CONFIG proxy.config.http.server_ports STRING 8080
CONFIG proxy.config.http.cache.http INT 1
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.url_remap.remap_required INT 1
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
And in remap.config:
reverse_map http://mediawiki.mycompany.com/<http://mediawiki.mycompany.com/> http://mediawiki.mycompany.com:8080/<http://mediawiki.mycompany.com:8080/>
I have never set up a proxy before.
Any suggestions on what I need to do? Thanks.
Re: Need help with a particular usage scenario
Posted by Alan Carroll <so...@yahoo-inc.com>.
Hmmm. A parent proxy set up might work better. The logging of empty URLs is a bug in the logging code which has been fixed (current ATS version is 6.0). If you are doing remap, you need a forward map as well like
map http://mediawiki.mycompany.com:8080/ http://mediawiki.mycompany/
reverse_map, if I recall correctly, only applies to response headers from the origin, to modify them to use the proxy URL instead of the origin URL (as supplied by the origin).
I'd recommend moving up in versions, to at least 5.3.x.
On Wednesday, January 20, 2016 12:23 PM, Victor Danilchenko <vd...@cimpress.com> wrote:
<!--#yiv3276921609 _filtered #yiv3276921609 {font-family:"Cambria Math";panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv3276921609 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;}#yiv3276921609 #yiv3276921609 p.yiv3276921609MsoNormal, #yiv3276921609 li.yiv3276921609MsoNormal, #yiv3276921609 div.yiv3276921609MsoNormal {margin:0in;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri", sans-serif;}#yiv3276921609 a:link, #yiv3276921609 span.yiv3276921609MsoHyperlink {color:#0563C1;text-decoration:underline;}#yiv3276921609 a:visited, #yiv3276921609 span.yiv3276921609MsoHyperlinkFollowed {color:#954F72;text-decoration:underline;}#yiv3276921609 span.yiv3276921609EmailStyle17 {font-family:"Calibri", sans-serif;color:windowtext;}#yiv3276921609 .yiv3276921609MsoChpDefault {font-family:"Calibri", sans-serif;} _filtered #yiv3276921609 {margin:1.0in 1.0in 1.0in 1.0in;}#yiv3276921609 div.yiv3276921609WordSection1 {}-->Hi all, I need to set up a caching proxy to not work as a proxy, but rather to simply provide caching upon direct access. For example: There’s a Mediawiki installation running at http://mediawiki.mycompany.com And I need to set up a proxy on an alternative port, http://mediawiki.mycompany.com:8080, which would do nothing but serve up the traffic from the default port, but cached (we aren’t trying to cache the entire MediaWiki installation, but rather provide an alternative entry point for a specific use case). I am having trouble figuring out what it would take to set up TrafficServer to do this. I set it up (Ubuntu Vivid, TS 3.2.4), but TS logs empty URLs (http:///) and of course fails to find them – presumably because it doesn’t receive the customary proxy HTTP headers; so I hit http://mediawiki.mycompany.com:8080, and only get back 404s. Is there an easy way to accomplish what I am looking to do? Here’s the config I have right now. In records.config (in addition to all the standard defaults): CONFIG proxy.config.http.server_ports STRING 8080 CONFIG proxy.config.http.cache.http INT 1 CONFIG proxy.config.reverse_proxy.enabled INT 1 CONFIG proxy.config.url_remap.remap_required INT 1 CONFIG proxy.config.url_remap.pristine_host_hdr INT 1 And in remap.config: reverse_map http://mediawiki.mycompany.com/ http://mediawiki.mycompany.com:8080/ I have never set up a proxy before. Any suggestions on what I need to do? Thanks.