You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Andrés Durán <du...@hotmail.com> on 2021/04/03 10:06:01 UTC
Problems Configuring ATS Layer 4 routing as outbound Proxy
Good morning guys, I would like to ask about ATS L4 routing, I'm trying to use it but I'm doing something wrong because ATS are not routing my HTTPS requests.
First of all I will put you in context:
- 1. ATS is compiled from source from its 9.0.1 tag using the next configuration line: './configure --prefix=/opt/ts --enable-curl --enable-wccp --enable-static --enable-experimental-plugins --enable-test-tools --with-ncurses --with-zlib --with-lzma --with-pcre --enable-posix-cap --enable-tproxy'
- 2. ATS 'sni.yaml' file is configrued as follows:
sni:
- fqdn: '*.debian.org'
tunnel_route: '$1.debian.org:443'
- 3. ATS 'ip_allow.yaml' file is configured as follows:
ip_allow:
- apply: in
ip_addrs: 127.0.0.1
action: allow
methods: ALL
- apply: in
ip_addrs: ::1
action: allow
methods: ALL
- apply: in
ip_addrs: 0/0
action: allow
methods:
- ALL
- apply: in
ip_addrs: ::/0
action: allow
methods:
- ALL
- 4. ATS 'records.config' file is configured as follows (default config lines are ommited):
CONFIG proxy.config.http.connect_ports STRING 0-8443
CONFIG proxy.config.url_remap.remap_required INT 0
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
CONFIG proxy.config.reverse_proxy.enabled INT 0
- 5. Client machine is a Debian 10 running a CURL command and configured as follows (ATS is 192.168.159.28):
export http_proxy=http://192.168.159.28:8080/
export https_proxy=http://192.168.159.28:8080/
curl -vvv https://www.debian.org
* Expire in 0 ms for 6 (transfer 0x5638bebcdfb0)
* Uses proxy env variable https_proxy == 'http://192.168.159.28:8080/'
* Trying 192.168.159.28...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5638bebcdfb0)
* Connected to 192.168.159.28 (192.168.159.28) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.debian.org:443
> CONNECT www.debian.org:443 HTTP/1.1
> Host: www.debian.org:443
> User-Agent: curl/7.64.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< Date: Sat, 03 Apr 2021 09:34:28 GMT
< Proxy-Connection: keep-alive
< Server: ATS/9.0.1
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to www.debian.org:443
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to www.debian.org:443
- 6. TCPDUMP on the ATS box shows this information (192.168.159.11 is client box and 192.168.159.28 is ATS box):
09:36:30.985034 IP 192.168.159.11.52670 > 192.168.159.28.8080: Flags [S], seq 1859333627, win 64240, options [mss 1460,sackOK,TS val 1425593008 ecr 0,nop,wscale 7], length 0
09:36:30.985169 IP 192.168.159.28.8080 > 192.168.159.11.52670: Flags [S.], seq 2335571830, ack 1859333628, win 65160, options [mss 1460,sackOK,TS val 968266333 ecr 1425593008,nop,wscale 7], length 0
09:36:30.985670 IP 192.168.159.11.52670 > 192.168.159.28.8080: Flags [.], ack 1, win 502, options [nop,nop,TS val 1425593009 ecr 968266333], length 0
09:36:30.986998 IP 192.168.159.11.52670 > 192.168.159.28.8080: Flags [P.], seq 1:121, ack 1, win 502, options [nop,nop,TS val 1425593010 ecr 968266333], length 120: HTTP: CONNECT www.debian.org:443 HTTP/1.1
09:36:30.987081 IP 192.168.159.28.8080 > 192.168.159.11.52670: Flags [.], ack 121, win 509, options [nop,nop,TS val 968266335 ecr 1425593010], length 0
09:36:30.991534 IP 192.168.159.28.59192 > 130.89.148.77.0: Flags [S], seq 2063572171, win 64240, options [mss 1460,sackOK,TS val 3451982116 ecr 0,nop,wscale 7], length 0
09:36:30.991988 IP 130.89.148.77.0 > 192.168.159.28.59192: Flags [R.], seq 0, ack 2063572172, win 64240, length 0
09:36:30.992133 IP 192.168.159.28.8080 > 192.168.159.11.52670: Flags [P.], seq 1:106, ack 121, win 509, options [nop,nop,TS val 968266340 ecr 1425593010], length 105: HTTP: HTTP/1.1 200 OK
09:36:30.992311 IP 192.168.159.28.8080 > 192.168.159.11.52670: Flags [F.], seq 106, ack 121, win 509, options [nop,nop,TS val 968266340 ecr 1425593010], length 0
09:36:30.992556 IP 192.168.159.11.52670 > 192.168.159.28.8080: Flags [.], ack 106, win 502, options [nop,nop,TS val 1425593015 ecr 968266340], length 0
09:36:30.998813 IP 192.168.159.11.52670 > 192.168.159.28.8080: Flags [P.], seq 121:638, ack 107, win 502, options [nop,nop,TS val 1425593022 ecr 968266340], length 517: HTTP
09:36:30.998888 IP 192.168.159.28.8080 > 192.168.159.11.52670: Flags [R], seq 2335571937, win 0, length 0
I do not understand why ATS is sending the packet outside '192.168.159.28.59192 > 130.89.148.77.0' using destination port '0', it is like ATS do not take in account the 'sni.yaml' configuration, but I see in ATS log file that 'sni.yaml' file is processed and loaded correctly. (No errors in ATS log files)
In the output firewall/router I see the error: 'Invalid transport field for protocol=TCP, from 192.168.159.28/59192 to 130.89.148.77/0'
Another question: Would be possible put ATS in full debug mode in order to analyze in detail what it is doing when the curl request are comming to ATS and for what modules of ATS the request is travelling?
Thank you guys in advance!
Warm regards.
Re: [E] Problems Configuring ATS Layer 4 routing as outbound Proxy
Posted by Alan Carroll <so...@verizonmedia.com>.
I can't find a 9.0.1 tag for Traffic Server - did you mean 9.0.1-rc0?
Testing with that I don't see the same problem - it works fine. Are you
running any plugins?
At this point, the best I can suggest is to enable the debug tag "http" and
let me see that output. It should contain the upstream IP address and port
to which ATS tries to connect.
Re: [E] Problems Configuring ATS Layer 4 routing as outbound Proxy
Posted by Alan Carroll <so...@verizonmedia.com>.
A few notes -
The SNI configuration will have no effect on this scenario. Here is a
diagram of the what happens when using an explicit proxy -
https://www.planttext.com/?text=LP11IyD048NlyoiUwLLBqPvoa6giA9P4I3n8o66JRDE5TLSckuR-UuS42xqCotnlthoTtHWKX_XjYLGNF2Fv73NZST0k950ZegBEky3U8gbO7O-cGwdvL_ECmqYYDE6Cv5ctHhcvsyzFegXm-o0QfCYAFDzd5QPfMYzuxNb8jzjx8X7Kgu6rTet85oeZLVQ1yYkVkVJ5BCNTPeFYdMXW7tyGedFQedwonlLuyJmfxeqRccLrlMfjrCr_0ciaAbxtXqD1pWSDDiCf22dpDvmqwnkqCK0GHic2zeph7yz1BgQNd5V6SewUQ2TLZle7
ATS never sees the "Client HELLO" as such, it's just bytes it forwards to
the upstream, and therefore never sees the SNI. SNI based configuration is
only useful if ATS is an implicit proxy.
Testing locally, your scenario works fine for me on master. I will try to
set up to test 9.0.1. Further testing was able to mimic your error by
selecting a port that is not in service on debian.org. Sadly, it appears
that ATS only goes so far as to send a SYN to the upstream - if that is
successful it responds with "200 OK". If the upstream doesn't like that SYN
(in my case, because there is nothing listening on that port) then it sends
a RESET but only after the user agent has sent its Client HELLO. ATS then
closes the connection to the user agent (because of the RESET) with the
result being the same as your reported error.
From what I can tell,"130.89.148.77" is a valid address for a Debian server
in Europe, so that seems reasonable.
The mystery here is why ATS apparently sends a packet with a zero port.
That shouldn't even make it past the next router, but perhaps that is what
is sending the RESET. I don't see that locally even using the same
"CONNECT" request.