You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Lei Sun <le...@gmail.com> on 2020/12/02 23:40:35 UTC
https issue
Hi Kit,
I'm trying to set up the trafficserver as an intermediary forward proxy.
For example,
1) http client send request to trafficserver.
2) trafficserver then pass this request to the downstream proxy
3) downstream proxy then route this request to the origin site
4) origin site send data back to the downstream proxy
5) downstream proxy send data back to trafficserver
6) trafficserver send data back to the http client.
I was able to make the entire request chain work if the origin site serves
content directly through HTTP.
> curl --proxy *http*://127.0.0.1:8080 *http*://httpbin.org/get?answer=4a -v
However, I ran into issues when I was trying to request for content served
from HTTPS.
$ curl --proxy *http*://127.0.0.1:8080 *https*://httpbin.org/get?answer=4a
> -v
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> * Establish HTTP proxy tunnel to httpbin.org:443
> > CONNECT httpbin.org:443 HTTP/1.1
> > Host: httpbin.org:443
> > User-Agent: curl/7.54.0
> > Proxy-Connection: Keep-Alive
> >
> < HTTP/1.1 200 OK
> < Date: Wed, 02 Dec 2020 20:53:31 GMT
> < Proxy-Connection: keep-alive
> < Server: ATS/10.0.0
> <
> * Proxy replied OK to CONNECT request
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/ssl/cert.pem
> CApath: none
> * *TLSv1.2 (OUT), TLS handshake, Client hello (1):*
> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
> * stopped the pause stream!
> * Closing connection 0
> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version
> number
>
From the error message, it seems that curl was able to connect to the
origin server, and even attempted to send the initial TLS handshake, but
somehow the handshake wasn't completed.
Here are my questions.
1) What's likely the cause?
2) How can I fix it.
Thank you!
Lei
Re: [E] https issue
Posted by Shu Kit Chan <ch...@gmail.com>.
That should be fine - simple dynamic hierarchical caching
If you are using hierarchical caching, you can write plugin using
https://docs.trafficserver.apache.org/en/latest/developer-guide/api/functions/TSHttpTxnParentProxySet.en.html
or
https://docs.trafficserver.apache.org/en/latest/developer-guide/api/functions/TSHttpTxnParentSelectionUrlGet.en.html
to control the parent proxy machines to go to.
We have the corresponding functions to call in lua as well.
If you are not using hierarchical caching, you can still use C or lua
plugin to control your destination, and thus forwarding the request
out to the upstream of your choice. And whether you do forward proxy
or reverse proxy, things should still be fine.
Perhaps you can share more about the your previous problem with this
curl command below
>
> $ curl --proxy http://127.0.0.1:8080 https://httpbin.org/get?answer=42 -v
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> * Establish HTTP proxy tunnel to httpbin.org:443
> > CONNECT httpbin.org:443 HTTP/1.1
> > Host: httpbin.org:443
> > User-Agent: curl/7.54.0
> > Proxy-Connection: Keep-Alive
> >
> < HTTP/1.1 200 OK
> < Date: Sun, 06 Dec 2020 02:22:49 GMT
> < Proxy-Connection: keep-alive
> < Server: ATS/10.0.0
> <
> * Proxy replied OK to CONNECT request
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/ssl/cert.pem
> CApath: none
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
> * stopped the pause stream!
> * Closing connection 0
> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
Are you using a lua script in this case? And if yes, what is the
content of the script?
On Mon, Dec 7, 2020 at 9:30 PM Lei Sun <le...@gmail.com> wrote:
>
> Hi Kit,
>
> It didn’t work.
>
> Do you think what I’m trying to do (dynamic hierarchical caching) possible with ATS?
>
> Thanks,
> Lei
>
> > On Dec 7, 2020, at 11:23 PM, Shu Kit Chan <ch...@gmail.com> wrote:
> >
> > Ok. I see. Thanks for explaining. So does my previous suggestion work then ?
> >
> >> On Sun, Dec 6, 2020 at 3:36 AM Lei Sun <le...@gmail.com> wrote:
> >>
> >> Hi Kit,
> >>
> >> Thanks for thinking through this with me.
> >>
> >> Here is what I'm trying to do.
> >>
> >> http/https client => proxy1 (ats) => proxy2_group (parent proxy group of 100 servers) => origin
> >>
> >> I would like to configure proxy1 (ats), so that it can forward all http/https requests through proxy2_group dynamically based on intelligent rules. For example, if node1, node2 and node3 of the proxy2_group is overloaded, the lua script can then direct more % of traffic to the other 97 nodes within the proxy2_group.
> >>
> >> It's very similar to the hierarchical caching, but dynamically with lua, since we'd literally be check hundreds of servers in real time.
> >>
> >> I'd like to figure out how to configure the ATS, so that it can do this.
> >>
> >> Thanks
> >> Lei
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>> On Sat, Dec 5, 2020 at 11:19 PM Shu Kit Chan <ch...@gmail.com> wrote:
> >>>
> >>> I am still not quite sure if i understand what you are trying to do.
> >>> But let me take a guess.
> >>>
> >>> The error "error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
> >>> version number" probably means that you are talking to a invalid https
> >>> endpoint through ATS. "https://httpbin.org/" is a pretty normal
> >>> destination. So are you already using lua plugin/script to change the
> >>> destination? That destination is probably not a valid https endpoint.
> >>> So the CONNECT may still work and return status 200 and follow the lua
> >>> script program to route request to new destination. That destination
> >>> is not valid https and so curl does not like it.
> >>>
> >>> If I am right so far, then I think disable the lua plugin will make
> >>> ATS to work just like squid. If you really want to use lua to change
> >>> the destination, I think you need to make sure to set the scheme to
> >>> https as well in your lua script.
> >>>
> >>> Thanks .
> >>>
> >>> Kit
> >>>
> >>>
> >>>
Re: [E] https issue
Posted by Lei Sun <le...@gmail.com>.
Hi Kit,
It didn’t work.
Do you think what I’m trying to do (dynamic hierarchical caching) possible with ATS?
Thanks,
Lei
> On Dec 7, 2020, at 11:23 PM, Shu Kit Chan <ch...@gmail.com> wrote:
>
> Ok. I see. Thanks for explaining. So does my previous suggestion work then ?
>
>> On Sun, Dec 6, 2020 at 3:36 AM Lei Sun <le...@gmail.com> wrote:
>>
>> Hi Kit,
>>
>> Thanks for thinking through this with me.
>>
>> Here is what I'm trying to do.
>>
>> http/https client => proxy1 (ats) => proxy2_group (parent proxy group of 100 servers) => origin
>>
>> I would like to configure proxy1 (ats), so that it can forward all http/https requests through proxy2_group dynamically based on intelligent rules. For example, if node1, node2 and node3 of the proxy2_group is overloaded, the lua script can then direct more % of traffic to the other 97 nodes within the proxy2_group.
>>
>> It's very similar to the hierarchical caching, but dynamically with lua, since we'd literally be check hundreds of servers in real time.
>>
>> I'd like to figure out how to configure the ATS, so that it can do this.
>>
>> Thanks
>> Lei
>>
>>
>>
>>
>>
>>
>>
>>> On Sat, Dec 5, 2020 at 11:19 PM Shu Kit Chan <ch...@gmail.com> wrote:
>>>
>>> I am still not quite sure if i understand what you are trying to do.
>>> But let me take a guess.
>>>
>>> The error "error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
>>> version number" probably means that you are talking to a invalid https
>>> endpoint through ATS. "https://httpbin.org/" is a pretty normal
>>> destination. So are you already using lua plugin/script to change the
>>> destination? That destination is probably not a valid https endpoint.
>>> So the CONNECT may still work and return status 200 and follow the lua
>>> script program to route request to new destination. That destination
>>> is not valid https and so curl does not like it.
>>>
>>> If I am right so far, then I think disable the lua plugin will make
>>> ATS to work just like squid. If you really want to use lua to change
>>> the destination, I think you need to make sure to set the scheme to
>>> https as well in your lua script.
>>>
>>> Thanks .
>>>
>>> Kit
>>>
>>>
>>>
Re: [E] https issue
Posted by Shu Kit Chan <ch...@gmail.com>.
Ok. I see. Thanks for explaining. So does my previous suggestion work then ?
On Sun, Dec 6, 2020 at 3:36 AM Lei Sun <le...@gmail.com> wrote:
>
> Hi Kit,
>
> Thanks for thinking through this with me.
>
> Here is what I'm trying to do.
>
> http/https client => proxy1 (ats) => proxy2_group (parent proxy group of 100 servers) => origin
>
> I would like to configure proxy1 (ats), so that it can forward all http/https requests through proxy2_group dynamically based on intelligent rules. For example, if node1, node2 and node3 of the proxy2_group is overloaded, the lua script can then direct more % of traffic to the other 97 nodes within the proxy2_group.
>
> It's very similar to the hierarchical caching, but dynamically with lua, since we'd literally be check hundreds of servers in real time.
>
> I'd like to figure out how to configure the ATS, so that it can do this.
>
> Thanks
> Lei
>
>
>
>
>
>
>
> On Sat, Dec 5, 2020 at 11:19 PM Shu Kit Chan <ch...@gmail.com> wrote:
>>
>> I am still not quite sure if i understand what you are trying to do.
>> But let me take a guess.
>>
>> The error "error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
>> version number" probably means that you are talking to a invalid https
>> endpoint through ATS. "https://httpbin.org/" is a pretty normal
>> destination. So are you already using lua plugin/script to change the
>> destination? That destination is probably not a valid https endpoint.
>> So the CONNECT may still work and return status 200 and follow the lua
>> script program to route request to new destination. That destination
>> is not valid https and so curl does not like it.
>>
>> If I am right so far, then I think disable the lua plugin will make
>> ATS to work just like squid. If you really want to use lua to change
>> the destination, I think you need to make sure to set the scheme to
>> https as well in your lua script.
>>
>> Thanks .
>>
>> Kit
>>
>>
>>
Re: [E] https issue
Posted by Lei Sun <le...@gmail.com>.
Hi Kit,
Thanks for thinking through this with me.
Here is what I'm trying to do.
http/https client => proxy1 (ats) => proxy2_group (parent proxy group of
100 servers) => origin
I would like to configure proxy1 (ats), so that it can forward all
http/https requests through proxy2_group dynamically based on intelligent
rules. For example, if node1, node2 and node3 of the proxy2_group is
overloaded, the lua script can then direct more % of traffic to the other
97 nodes within the proxy2_group.
It's very similar to the hierarchical caching
<https://docs.trafficserver.apache.org/en/latest/admin-guide/configuration/hierarchical-caching.en.html>,
but dynamically with lua, since we'd literally be check hundreds of servers
in real time.
I'd like to figure out how to configure the ATS, so that it can do this.
Thanks
Lei
On Sat, Dec 5, 2020 at 11:19 PM Shu Kit Chan <ch...@gmail.com> wrote:
> I am still not quite sure if i understand what you are trying to do.
> But let me take a guess.
>
> The error "error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
> version number" probably means that you are talking to a invalid https
> endpoint through ATS. "https://httpbin.org/" is a pretty normal
> destination. So are you already using lua plugin/script to change the
> destination? That destination is probably not a valid https endpoint.
> So the CONNECT may still work and return status 200 and follow the lua
> script program to route request to new destination. That destination
> is not valid https and so curl does not like it.
>
> If I am right so far, then I think disable the lua plugin will make
> ATS to work just like squid. If you really want to use lua to change
> the destination, I think you need to make sure to set the scheme to
> https as well in your lua script.
>
> Thanks .
>
> Kit
>
>
>
>
Re: [E] https issue
Posted by Shu Kit Chan <ch...@gmail.com>.
I am still not quite sure if i understand what you are trying to do.
But let me take a guess.
The error "error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
version number" probably means that you are talking to a invalid https
endpoint through ATS. "https://httpbin.org/" is a pretty normal
destination. So are you already using lua plugin/script to change the
destination? That destination is probably not a valid https endpoint.
So the CONNECT may still work and return status 200 and follow the lua
script program to route request to new destination. That destination
is not valid https and so curl does not like it.
If I am right so far, then I think disable the lua plugin will make
ATS to work just like squid. If you really want to use lua to change
the destination, I think you need to make sure to set the scheme to
https as well in your lua script.
Thanks .
Kit
On Sat, Dec 5, 2020 at 6:25 PM Lei Sun <le...@gmail.com> wrote:
>
> Hi Kit,
>
> I set up the ATS to be a forward proxy, so I turned remap off.
>>
>> CONFIG proxy.config.url_remap.remap_required INT 0
>> CONFIG proxy.config.reverse_proxy.enabled INT 0
>
>
> I was trying to use lua scripts to intercept the incoming request, and passing the request to upstream/parent cache proxy dynamically based on some custom logic.
> I was able to make the http work by intercepting and modifying request, url, server_request.server_addr, etc. in do_global_post_remap and do_global_send_request hooks. However, it threw the following error when I tried https
>>
>> $ curl --proxy http://127.0.0.1:8080 https://httpbin.org/get?answer=42 -v
>> * Trying 127.0.0.1...
>> * TCP_NODELAY set
>> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
>> * Establish HTTP proxy tunnel to httpbin.org:443
>> > CONNECT httpbin.org:443 HTTP/1.1
>> > Host: httpbin.org:443
>> > User-Agent: curl/7.54.0
>> > Proxy-Connection: Keep-Alive
>> >
>> < HTTP/1.1 200 OK
>> < Date: Sun, 06 Dec 2020 02:22:49 GMT
>> < Proxy-Connection: keep-alive
>> < Server: ATS/10.0.0
>> <
>> * Proxy replied OK to CONNECT request
>> * ALPN, offering h2
>> * ALPN, offering http/1.1
>> * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>> * successfully set certificate verify locations:
>> * CAfile: /etc/ssl/cert.pem
>> CApath: none
>> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
>> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
>> * stopped the pause stream!
>> * Closing connection 0
>> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
>
>
> I was able to get the desired request forwarding functionality working using squid's cache_peer feature, see below
> curl --proxy http://127.0.0.1:3128 http://httpbin.org/get?answer=42 -v
> curl --proxy http://127.0.0.1:3128 https://httpbin.org/get?answer=42 -v
>
> However, squid only supports hard coded configuration of cache_peer directive.
>>
>> cache_peer 23.105.0.211 parent 29842 0 no-query no-digest login=USERNAME:PASSWORD
>
>
> Could you please give me some advice on what to modify in lua so that we can get the same functionality as squid, except that we can do it dynamically with custom rules, which is a lot more powerful.
>
> Cheers,
> Lei
>
> On Sat, Dec 5, 2020 at 4:42 PM Shu Kit Chan <ch...@gmail.com> wrote:
>>
>> do_global_send_request in lua is the function called for the
>> TS_HTTP_SEND_REQUEST_HDR_HOOK -
>> https://docs.trafficserver.apache.org/en/latest/developer-guide/plugins/hooks-and-transactions/adding-hooks.en.html
>>
>> You are getting a 502 from ATS for this -
>> curl -k -H "Host: httbin.org:443" https://127.0.0.1:8443 -vv" .
>>
>> So what does your remap.config looks like?
>> For the above to work, you will need a remap rule to map httbin.org
>> (not httpbin.org?) to somewhere. It looks like ATS is not able to
>> connect to that "somewhere" and thus if you have a
>> "do_global_send_request" function in your lua script, it won't get
>> executed.
>>
>> Kit
>>
Re: [E] https issue
Posted by Lei Sun <le...@gmail.com>.
Hi Kit,
I set up the ATS to be a forward proxy, so I turned remap off.
> CONFIG proxy.config.url_remap.remap_required INT 0
> CONFIG proxy.config.reverse_proxy.enabled INT 0
I was trying to use lua scripts to intercept the incoming request, and
passing the request to upstream/parent cache proxy dynamically based on
some custom logic.
I was able to make the http work by intercepting and modifying request,
url, server_request.server_addr, etc. in *do_global_post_remap* and
*do_global_send_request
*hooks. However, it threw the following error when I tried https
> $ curl --proxy http://127.0.0.1:8080 https://httpbin.org/get?answer=42 -v
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> * Establish HTTP proxy tunnel to httpbin.org:443
> > CONNECT httpbin.org:443 HTTP/1.1
> > Host: httpbin.org:443
> > User-Agent: curl/7.54.0
> > Proxy-Connection: Keep-Alive
> >
> < HTTP/1.1 200 OK
> < Date: Sun, 06 Dec 2020 02:22:49 GMT
> < Proxy-Connection: keep-alive
> < Server: ATS/10.0.0
> <
> * Proxy replied OK to CONNECT request
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/ssl/cert.pem
> CApath: none
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
> * stopped the pause stream!
> * Closing connection 0
> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version
> number
I was able to get the desired request forwarding functionality working
using squid's cache_peer feature, see below
curl --proxy http://127.0.0.1:3128 http://httpbin.org/get?answer=42 -v
curl --proxy http://127.0.0.1:3128 https://httpbin.org/get?answer=42 -v
However, squid only supports hard coded configuration of cache_peer
directive.
> cache_peer 23.105.0.211 parent 29842 0 no-query no-digest
> login=USERNAME:PASSWORD
Could you please give me some advice on what to modify in lua so that we
can get the same functionality as squid, except that we can do it
dynamically with custom rules, which is a lot more powerful.
Cheers,
Lei
On Sat, Dec 5, 2020 at 4:42 PM Shu Kit Chan <ch...@gmail.com> wrote:
> do_global_send_request in lua is the function called for the
> TS_HTTP_SEND_REQUEST_HDR_HOOK -
>
> https://docs.trafficserver.apache.org/en/latest/developer-guide/plugins/hooks-and-transactions/adding-hooks.en.html
>
> You are getting a 502 from ATS for this -
> curl -k -H "Host: httbin.org:443" https://127.0.0.1:8443 -vv" .
>
> So what does your remap.config looks like?
> For the above to work, you will need a remap rule to map httbin.org
> (not httpbin.org?) to somewhere. It looks like ATS is not able to
> connect to that "somewhere" and thus if you have a
> "do_global_send_request" function in your lua script, it won't get
> executed.
>
> Kit
>
>
Re: [E] https issue
Posted by Shu Kit Chan <ch...@gmail.com>.
do_global_send_request in lua is the function called for the
TS_HTTP_SEND_REQUEST_HDR_HOOK -
https://docs.trafficserver.apache.org/en/latest/developer-guide/plugins/hooks-and-transactions/adding-hooks.en.html
You are getting a 502 from ATS for this -
curl -k -H "Host: httbin.org:443" https://127.0.0.1:8443 -vv" .
So what does your remap.config looks like?
For the above to work, you will need a remap rule to map httbin.org
(not httpbin.org?) to somewhere. It looks like ATS is not able to
connect to that "somewhere" and thus if you have a
"do_global_send_request" function in your lua script, it won't get
executed.
Kit
On Sat, Dec 5, 2020 at 12:31 PM Lei Sun <le...@gmail.com> wrote:
>
> It seems whenever https was the end URL, do_global_send_request is never called.
>
> Is it possible to force the above execution path for https?
>
> Essentially, I'm looking for a way for the ATS (dynamically through lua) to hand over the request to a parent proxy.
>
> Thanks,
> Lei
>
> On Sat, Dec 5, 2020 at 2:20 PM Lei Sun <le...@gmail.com> wrote:
>>
>> Hi Alan,
>>
>> Yes, great point!
>>
>> Here are the output for the latest recommandations
>>
>>> $ curl -H "Host: httbin.org:443" https://127.0.0.1:8443 -vv
>>> * Rebuilt URL to: https://127.0.0.1:8443/
>>> * Trying 127.0.0.1...
>>> * TCP_NODELAY set
>>> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
>>> * ALPN, offering h2
>>> * ALPN, offering http/1.1
>>> * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>>> * successfully set certificate verify locations:
>>> * CAfile: /etc/ssl/cert.pem
>>> CApath: none
>>> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
>>> * TLSv1.2 (IN), TLS handshake, Server hello (2):
>>> * TLSv1.2 (IN), TLS handshake, Certificate (11):
>>> * TLSv1.2 (OUT), TLS alert, Server hello (2):
>>> * SSL certificate problem: self signed certificate
>>> * stopped the pause stream!
>>> * Closing connection 0
>>> curl: (60) SSL certificate problem: self signed certificate
>>> More details here: https://curl.haxx.se/docs/sslcerts.html
>>> curl performs SSL certificate verification by default, using a "bundle"
>>> of Certificate Authority (CA) public keys (CA certs). If the default
>>> bundle file isn't adequate, you can specify an alternate file
>>> using the --cacert option.
>>> If this HTTPS server uses a certificate signed by a CA represented in
>>> the bundle, the certificate verification probably failed due to a
>>> problem with the certificate (it might be expired, or the name might
>>> not match the domain name in the URL).
>>> If you'd like to turn off curl's verification of the certificate, use
>>> the -k (or --insecure) option.
>>> HTTPS-proxy has similar options --proxy-cacert and --proxy-insecure.
>>
>>
>> Then I tried
>>>
>>> $ curl -k -H "Host: httbin.org:443" https://127.0.0.1:8443 -vv
>>> * Rebuilt URL to: https://127.0.0.1:8443/
>>> * Trying 127.0.0.1...
>>> * TCP_NODELAY set
>>> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
>>> * ALPN, offering h2
>>> * ALPN, offering http/1.1
>>> * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>>> * successfully set certificate verify locations:
>>> * CAfile: /etc/ssl/cert.pem
>>> CApath: none
>>> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
>>> * TLSv1.2 (IN), TLS handshake, Server hello (2):
>>> * TLSv1.2 (IN), TLS handshake, Certificate (11):
>>> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
>>> * TLSv1.2 (IN), TLS handshake, Server finished (14):
>>> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
>>> * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
>>> * TLSv1.2 (OUT), TLS handshake, Finished (20):
>>> * TLSv1.2 (IN), TLS change cipher, Client hello (1):
>>> * TLSv1.2 (IN), TLS handshake, Finished (20):
>>> * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
>>> * ALPN, server accepted to use h2
>>> * Server certificate:
>>> * subject: C=US
>>> * start date: Dec 5 04:41:08 2020 GMT
>>> * expire date: Dec 3 04:41:08 2030 GMT
>>> * issuer: C=US
>>> * SSL certificate verify result: self signed certificate (18), continuing anyway.
>>> * Using HTTP2, server supports multi-use
>>> * Connection state changed (HTTP/2 confirmed)
>>> * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
>>> * Using Stream ID: 1 (easy handle 0x7f9cf9006600)
>>> > GET / HTTP/2
>>> > Host: httbin.org:443
>>> > User-Agent: curl/7.54.0
>>> > Accept: */*
>>> >
>>> * Connection state changed (MAX_CONCURRENT_STREAMS updated)!
>>> < HTTP/2 502
>>> < date: Sat, 05 Dec 2020 20:16:39 GMT
>>> < server: ATS/10.0.0
>>> < cache-control: no-store
>>> < content-type: text/html
>>> < content-language: en
>>> < content-length: 247
>>> <
>>> <HTML>
>>> <HEAD>
>>> <TITLE>Could Not Connect</TITLE>
>>> </HEAD>
>>> <BODY BGCOLOR="white" FGCOLOR="black">
>>> <H1>Could Not Connect</H1>
>>> <HR>
>>> <FONT FACE="Helvetica,Arial"><B>
>>> Description: Could not connect to the requested server host.
>>> </B></FONT>
>>> <HR>
>>> </BODY>
>>> * Connection #0 to host 127.0.0.1 left intact
>>
>>
>>> $ curl -k --proxy-insecure --proxy https://127.0.0.1:8443 https://httpbin.org:8443/get?answer=42 -v
>>> * Trying 127.0.0.1...
>>> * TCP_NODELAY set
>>> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
>>> * ALPN, offering h2
>>> * ALPN, offering http/1.1
>>> * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>>> * successfully set certificate verify locations:
>>> * CAfile: /etc/ssl/cert.pem
>>> CApath: none
>>> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
>>> * TLSv1.2 (IN), TLS handshake, Server hello (2):
>>> * TLSv1.2 (IN), TLS handshake, Certificate (11):
>>> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
>>> * TLSv1.2 (IN), TLS handshake, Server finished (14):
>>> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
>>> * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
>>> * TLSv1.2 (OUT), TLS handshake, Finished (20):
>>> * TLSv1.2 (IN), TLS change cipher, Client hello (1):
>>> * TLSv1.2 (IN), TLS handshake, Finished (20):
>>> * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
>>> * ALPN, server accepted to use h2
>>> * Proxy certificate:
>>> * subject: C=US
>>> * start date: Dec 5 04:41:08 2020 GMT
>>> * expire date: Dec 3 04:41:08 2030 GMT
>>> * issuer: C=US
>>> * SSL certificate verify result: self signed certificate (18), continuing anyway.
>>> * Establish HTTP proxy tunnel to httpbin.org:8443
>>> > CONNECT httpbin.org:8443 HTTP/1.1
>>> > Host: httpbin.org:8443
>>> > User-Agent: curl/7.54.0
>>> > Proxy-Connection: Keep-Alive
>>> >
>>> * TLSv1.2 (IN), TLS alert, Client hello (1):
>>> * Proxy CONNECT aborted
>>> * Connection #0 to host 127.0.0.1 left intact
>>> curl: (56) Proxy CONNECT aborted
>>
>>
>> Hi Guys, if anyone has a few minutes, happy to hop on a zoom, and share my screen, where we can quickly try out different options?
>>
>> Thanks,
>> Lei
>>
>>
>
>
> --
> Stay Hungry, Stay Foolish.
> Lei Sun
> Cell: 408-306-9199
Re: [E] https issue
Posted by Lei Sun <le...@gmail.com>.
It seems whenever https was the end URL, *do_global_send_request* is never
called.
Is it possible to force the above execution path for https?
Essentially, I'm looking for a way for the ATS (dynamically through lua) to
hand over the request to a parent proxy.
Thanks,
Lei
On Sat, Dec 5, 2020 at 2:20 PM Lei Sun <le...@gmail.com> wrote:
> Hi Alan,
>
> Yes, great point!
>
> Here are the output for the latest recommandations
>
> *$ curl -H "Host: httbin.org:443 <http://httbin.org:443>"
>> https://127.0.0.1:8443 <https://127.0.0.1:8443> -vv*
>> * Rebuilt URL to: https://127.0.0.1:8443/
>> * Trying 127.0.0.1...
>> * TCP_NODELAY set
>> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
>> * ALPN, offering h2
>> * ALPN, offering http/1.1
>> * Cipher selection:
>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>> * successfully set certificate verify locations:
>> * CAfile: /etc/ssl/cert.pem
>> CApath: none
>> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
>> * TLSv1.2 (IN), TLS handshake, Server hello (2):
>> * TLSv1.2 (IN), TLS handshake, Certificate (11):
>> * TLSv1.2 (OUT), TLS alert, Server hello (2):
>> * SSL certificate problem: self signed certificate
>> * stopped the pause stream!
>> * Closing connection 0
>> curl: (60) SSL certificate problem: self signed certificate
>> More details here: https://curl.haxx.se/docs/sslcerts.html
>> curl performs SSL certificate verification by default, using a "bundle"
>> of Certificate Authority (CA) public keys (CA certs). If the default
>> bundle file isn't adequate, you can specify an alternate file
>> using the --cacert option.
>> If this HTTPS server uses a certificate signed by a CA represented in
>> the bundle, the certificate verification probably failed due to a
>> problem with the certificate (it might be expired, or the name might
>> not match the domain name in the URL).
>> If you'd like to turn off curl's verification of the certificate, use
>> the -k (or --insecure) option.
>> HTTPS-proxy has similar options --proxy-cacert and --proxy-insecure.
>
>
> Then I tried
>
>> $ *curl -k -H "Host: httbin.org:443 <http://httbin.org:443>"
>> https://127.0.0.1:8443 <https://127.0.0.1:8443> -vv*
>> * Rebuilt URL to: https://127.0.0.1:8443/
>> * Trying 127.0.0.1...
>> * TCP_NODELAY set
>> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
>> * ALPN, offering h2
>> * ALPN, offering http/1.1
>> * Cipher selection:
>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>> * successfully set certificate verify locations:
>> * CAfile: /etc/ssl/cert.pem
>> CApath: none
>> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
>> * TLSv1.2 (IN), TLS handshake, Server hello (2):
>> * TLSv1.2 (IN), TLS handshake, Certificate (11):
>> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
>> * TLSv1.2 (IN), TLS handshake, Server finished (14):
>> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
>> * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
>> * TLSv1.2 (OUT), TLS handshake, Finished (20):
>> * TLSv1.2 (IN), TLS change cipher, Client hello (1):
>> * TLSv1.2 (IN), TLS handshake, Finished (20):
>> * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
>> * ALPN, server accepted to use h2
>> * Server certificate:
>> * subject: C=US
>> * start date: Dec 5 04:41:08 2020 GMT
>> * expire date: Dec 3 04:41:08 2030 GMT
>> * issuer: C=US
>> * SSL certificate verify result: self signed certificate (18),
>> continuing anyway.
>> * Using HTTP2, server supports multi-use
>> * Connection state changed (HTTP/2 confirmed)
>> * Copying HTTP/2 data in stream buffer to connection buffer after
>> upgrade: len=0
>> * Using Stream ID: 1 (easy handle 0x7f9cf9006600)
>> > GET / HTTP/2
>> > Host: httbin.org:443
>> > User-Agent: curl/7.54.0
>> > Accept: */*
>> >
>> * Connection state changed (MAX_CONCURRENT_STREAMS updated)!
>> < HTTP/2 502
>> < date: Sat, 05 Dec 2020 20:16:39 GMT
>> < server: ATS/10.0.0
>> < cache-control: no-store
>> < content-type: text/html
>> < content-language: en
>> < content-length: 247
>> <
>> <HTML>
>> <HEAD>
>> <TITLE>Could Not Connect</TITLE>
>> </HEAD>
>> <BODY BGCOLOR="white" FGCOLOR="black">
>> <H1>Could Not Connect</H1>
>> <HR>
>> <FONT FACE="Helvetica,Arial"><B>
>> Description: Could not connect to the requested server host.
>> </B></FONT>
>> <HR>
>> </BODY>
>> * Connection #0 to host 127.0.0.1 left intact
>
>
> $ *curl -k --proxy-insecure --proxy https://127.0.0.1:8443
>> <https://127.0.0.1:8443> https://httpbin.org:8443/get?answer=42
>> <https://httpbin.org:8443/get?answer=42> -v*
>> * Trying 127.0.0.1...
>> * TCP_NODELAY set
>> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
>> * ALPN, offering h2
>> * ALPN, offering http/1.1
>> * Cipher selection:
>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>> * successfully set certificate verify locations:
>> * CAfile: /etc/ssl/cert.pem
>> CApath: none
>> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
>> * TLSv1.2 (IN), TLS handshake, Server hello (2):
>> * TLSv1.2 (IN), TLS handshake, Certificate (11):
>> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
>> * TLSv1.2 (IN), TLS handshake, Server finished (14):
>> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
>> * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
>> * TLSv1.2 (OUT), TLS handshake, Finished (20):
>> * TLSv1.2 (IN), TLS change cipher, Client hello (1):
>> * TLSv1.2 (IN), TLS handshake, Finished (20):
>> * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
>> * ALPN, server accepted to use h2
>> * Proxy certificate:
>> * subject: C=US
>> * start date: Dec 5 04:41:08 2020 GMT
>> * expire date: Dec 3 04:41:08 2030 GMT
>> * issuer: C=US
>> * SSL certificate verify result: self signed certificate (18),
>> continuing anyway.
>> * Establish HTTP proxy tunnel to httpbin.org:8443
>> > CONNECT httpbin.org:8443 HTTP/1.1
>> > Host: httpbin.org:8443
>> > User-Agent: curl/7.54.0
>> > Proxy-Connection: Keep-Alive
>> >
>> * TLSv1.2 (IN), TLS alert, Client hello (1):
>> * Proxy CONNECT aborted
>> * Connection #0 to host 127.0.0.1 left intact
>> curl: (56) Proxy CONNECT aborted
>
>
> Hi Guys, if anyone has a few minutes, happy to hop on a zoom, and share my
> screen, where we can quickly try out different options?
>
> Thanks,
> Lei
>
>
>
--
Stay Hungry, Stay Foolish.
Lei Sun
Cell: 408-306-9199
Re: [E] https issue
Posted by Lei Sun <le...@gmail.com>.
Hi Alan,
Yes, great point!
Here are the output for the latest recommandations
*$ curl -H "Host: httbin.org:443 <http://httbin.org:443>"
> https://127.0.0.1:8443 <https://127.0.0.1:8443> -vv*
> * Rebuilt URL to: https://127.0.0.1:8443/
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/ssl/cert.pem
> CApath: none
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (OUT), TLS alert, Server hello (2):
> * SSL certificate problem: self signed certificate
> * stopped the pause stream!
> * Closing connection 0
> curl: (60) SSL certificate problem: self signed certificate
> More details here: https://curl.haxx.se/docs/sslcerts.html
> curl performs SSL certificate verification by default, using a "bundle"
> of Certificate Authority (CA) public keys (CA certs). If the default
> bundle file isn't adequate, you can specify an alternate file
> using the --cacert option.
> If this HTTPS server uses a certificate signed by a CA represented in
> the bundle, the certificate verification probably failed due to a
> problem with the certificate (it might be expired, or the name might
> not match the domain name in the URL).
> If you'd like to turn off curl's verification of the certificate, use
> the -k (or --insecure) option.
> HTTPS-proxy has similar options --proxy-cacert and --proxy-insecure.
Then I tried
> $ *curl -k -H "Host: httbin.org:443 <http://httbin.org:443>"
> https://127.0.0.1:8443 <https://127.0.0.1:8443> -vv*
> * Rebuilt URL to: https://127.0.0.1:8443/
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/ssl/cert.pem
> CApath: none
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> * TLSv1.2 (IN), TLS handshake, Server finished (14):
> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
> * TLSv1.2 (OUT), TLS handshake, Finished (20):
> * TLSv1.2 (IN), TLS change cipher, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Finished (20):
> * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
> * ALPN, server accepted to use h2
> * Server certificate:
> * subject: C=US
> * start date: Dec 5 04:41:08 2020 GMT
> * expire date: Dec 3 04:41:08 2030 GMT
> * issuer: C=US
> * SSL certificate verify result: self signed certificate (18), continuing
> anyway.
> * Using HTTP2, server supports multi-use
> * Connection state changed (HTTP/2 confirmed)
> * Copying HTTP/2 data in stream buffer to connection buffer after upgrade:
> len=0
> * Using Stream ID: 1 (easy handle 0x7f9cf9006600)
> > GET / HTTP/2
> > Host: httbin.org:443
> > User-Agent: curl/7.54.0
> > Accept: */*
> >
> * Connection state changed (MAX_CONCURRENT_STREAMS updated)!
> < HTTP/2 502
> < date: Sat, 05 Dec 2020 20:16:39 GMT
> < server: ATS/10.0.0
> < cache-control: no-store
> < content-type: text/html
> < content-language: en
> < content-length: 247
> <
> <HTML>
> <HEAD>
> <TITLE>Could Not Connect</TITLE>
> </HEAD>
> <BODY BGCOLOR="white" FGCOLOR="black">
> <H1>Could Not Connect</H1>
> <HR>
> <FONT FACE="Helvetica,Arial"><B>
> Description: Could not connect to the requested server host.
> </B></FONT>
> <HR>
> </BODY>
> * Connection #0 to host 127.0.0.1 left intact
$ *curl -k --proxy-insecure --proxy https://127.0.0.1:8443
> <https://127.0.0.1:8443> https://httpbin.org:8443/get?answer=42
> <https://httpbin.org:8443/get?answer=42> -v*
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/ssl/cert.pem
> CApath: none
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> * TLSv1.2 (IN), TLS handshake, Server finished (14):
> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
> * TLSv1.2 (OUT), TLS handshake, Finished (20):
> * TLSv1.2 (IN), TLS change cipher, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Finished (20):
> * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
> * ALPN, server accepted to use h2
> * Proxy certificate:
> * subject: C=US
> * start date: Dec 5 04:41:08 2020 GMT
> * expire date: Dec 3 04:41:08 2030 GMT
> * issuer: C=US
> * SSL certificate verify result: self signed certificate (18), continuing
> anyway.
> * Establish HTTP proxy tunnel to httpbin.org:8443
> > CONNECT httpbin.org:8443 HTTP/1.1
> > Host: httpbin.org:8443
> > User-Agent: curl/7.54.0
> > Proxy-Connection: Keep-Alive
> >
> * TLSv1.2 (IN), TLS alert, Client hello (1):
> * Proxy CONNECT aborted
> * Connection #0 to host 127.0.0.1 left intact
> curl: (56) Proxy CONNECT aborted
Hi Guys, if anyone has a few minutes, happy to hop on a zoom, and share my
screen, where we can quickly try out different options?
Thanks,
Lei
Re: [E] https issue
Posted by Alan Carroll <so...@verizonmedia.com>.
Did you try this
*curl -H "Host: httbin.org:443
<https://urldefense.proofpoint.com/v2/url?u=http-3A__httbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=KGt-kuF-04DSoLQysb_TMJDLA4T9KCKYMuLArHdTLas&s=MBLXeQRLg_U6jGJDITRJv1XDNrdcKUXXNLdoJw747QE&e=>"
https://127.0.0.1:8443
<https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=KGt-kuF-04DSoLQysb_TMJDLA4T9KCKYMuLArHdTLas&s=KOw_NDzToZ5m9aOAknPwadHs8jMEx_YKDAWaT5kKRX4&e=>
-vv*
Note the "https" for the URL. If you don't use "--proxy" and the URL is
"http", then curl will not attempt to use TLS during the connection.
Alternatively, note Susan's suggestion -
curl -k -v --resolve 'httbin.org:4443:127.0.0.1'
https://httpbin.org:4443/get?answer=4a
<https://urldefense.proofpoint.com/v2/url?u=https-3A__httpbin.org-3A4443_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=CJRs6tsuoI10rfl5-7BXCVEZV1qNbJIXaf3CblZVIjU&s=_9FxWsURWRJuQDxVN0_htTFCut7inhRRStAJHtgPEbA&e=>
What you actually did was
curl -k -v --resolve 'httbin.org:8443:127.0.0.1'
https://httpbin.org/get?answer=42
<https://urldefense.proofpoint.com/v2/url?u=https-3A__httpbin.org_get-3Fanswer-3D42&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=GCeGAsAuMOtYawkiZoW91LT001Dp0_yEVsYttQWDsac&s=pygV3JDTpp005eNPhaNU8vn3U6hWMBTvZl-sIpm82OI&e=>
-v
These are not the same.
Re: [E] https issue
Posted by Lei Sun <le...@gmail.com>.
Hi Guys,
I was able to figure out how to do this with Squid's cache_peer feature.
> cache_peer 23.105.0.211 parent 29842 0 no-query no-digest
> login=USERNAME:PASSWORD
> never_direct allow all
Squid was able to essentially forward my request to *23.105.0.211*, plus
the basic auth, and return the result
> $ curl --proxy http://127.0.0.1:3128 https://httpbin.org/get?answer=42
> {
> "args": {
> "answer": "42"
> },
> "headers": {
> "Accept": "*/*",
> "Host": "httpbin.org",
> "User-Agent": "curl/7.54.0",
> "X-Amzn-Trace-Id": "Root=1-5fcbd8a7-5a21842b590a9e1637bdc520"
> },
> "origin": "*23.105.0.211*",
> "url": "https://httpbin.org/get?answer=42"
> }
Question:
1) How can I make ATS do the same thing?
2) Can I use a lua script to do this? (I was able to do this with a normal
forward proxy configuration plus a lua script to rewrite headers & next
host address for normal http successfully, but failing for https requests
somehow. What should be done so that this could work, please?)
Thank you!
Lei
Re: [E] https issue
Posted by Lei Sun <le...@gmail.com>.
Hi,
I gave these a try, and none of these worked. Please see the output below
> *$ curl -H "Host: httbin.org <http://httbin.org>" http://127.0.0.1:8443
> <http://127.0.0.1:8443> -vv*
> * Rebuilt URL to: http://127.0.0.1:8443/
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
> > GET / HTTP/1.1
> > Host: httbin.org
> > User-Agent: curl/7.54.0
> > Accept: */*
> >
> * Empty reply from server
> * Connection #0 to host 127.0.0.1 left intact
> curl: (52) Empty reply from server
*$ curl -H "Host: httbin.org:443 <http://httbin.org:443>"
> http://127.0.0.1:8443 <http://127.0.0.1:8443> -vv*
> * Rebuilt URL to: http://127.0.0.1:8443/
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
> > GET / HTTP/1.1
> > Host: httbin.org:443
> > User-Agent: curl/7.54.0
> > Accept: */*
> >
> * Empty reply from server
> * Connection #0 to host 127.0.0.1 left intact
> curl: (52) Empty reply from server
*curl --proxy http://127.0.0.1:8443 <http://127.0.0.1:8443>
> https://httbin.org <https://httbin.org> -vv*
> * Rebuilt URL to: https://httbin.org/
> * No valid port number in proxy string (8443 )
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to 127.0.0.1 (127.0.0.1) port 8443 (#0)
> * Establish HTTP proxy tunnel to httbin.org:443
> > CONNECT httbin.org:443 HTTP/1.1
> > Host: httbin.org:443
> > User-Agent: curl/7.54.0
> > Proxy-Connection: Keep-Alive
> >
> * Proxy CONNECT aborted
> * Connection #0 to host 127.0.0.1 left intact
> curl: (56) Proxy CONNECT aborted
On Sat, Dec 5, 2020 at 2:13 AM pentester <pe...@cert.gov.az> wrote:
>
> Hello,
>
> Give a try please
>
> 1) curl -H "Host: httbin.org" http://127.0.0.1:8443 -vv
> 2) curl -H "Host: httbin.org:443" http://127.0.0.1:8443 -vv
> 3) curl --proxy http://127.0.0.1:8443 https://httbin.org -vv
>
> Should work but can't confirm it against ATS since i can't reach my
> test machine right now. Sorry.
>
>
>
Re: [E] https issue
Posted by pentester <pe...@cert.gov.az>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello,
Give a try please
1) curl -H "Host: httbin.org" http://127.0.0.1:8443 -vv
2) curl -H "Host: httbin.org:443" http://127.0.0.1:8443 -vv
3) curl --proxy http://127.0.0.1:8443 https://httbin.org -vv
Should work but can't confirm it against ATS since i can't reach my
test machine right now. Sorry.
- ------------------------------
On Sat, 2020-12-05 at 00:18 -0600, Lei Sun wrote:
> Hi Guys,
>
> I followed the instructions from Alan, and set up the certs.
>
> > dest_ip=* ssl_cert_name=host.pem ssl_key_name=host.key
>
>
> then I followed the instructions from Susan and tested with --resolve
>
> curl -k -v --resolve 'httbin.org:8443:127.0.0.1'
> > https://httpbin.org/get?answer=42 -v
>
>
> It seems that the ats still lets curl create tls handshake directly
> with
> the origin server, which isn't what I'm looking for.
>
> > $ curl -k -v --resolve 'httbin.org:8443:127.0.0.1'
> > https://httpbin.org/get?answer=42 -v
> > * Added httbin.org:8443:127.0.0.1 to DNS cache
> > * Trying 184.72.216.47...
> > * TCP_NODELAY set
> > * Connected to httpbin.org (184.72.216.47) port 443 (#0)
> > * ALPN, offering h2
> > * ALPN, offering http/1.1
> > * Cipher selection:
> > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > * successfully set certificate verify locations:
> > * CAfile: /etc/ssl/cert.pem
> > CApath: none
> > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > * TLSv1.2 (IN), TLS handshake, Server hello (2):
> > * TLSv1.2 (IN), TLS handshake, Certificate (11):
>
>
> I'm trying to do the following, notice the "origin":
> "*103.253.27.108*",
> except that I want to make the request to ATS on *127.0.0.1*, then
> have the
> ATS forward this request to *103.253.27.108*, and return the *https*
> response from *103.253.27.108* back to the client. How can we make
> this
> happen with ATS please?
>
> $ curl --proxy http://103.253.27.108:80
> *https*://httpbin.org/get?answer=42
> > -v
> > * Trying 103.253.27.108...
> > * TCP_NODELAY set
> > * Connected to 103.253.27.108 (103.253.27.108) port 80 (#0)
> > * Establish HTTP proxy tunnel to httpbin.org:443
> > > CONNECT httpbin.org:443 HTTP/1.1
> > > Host: httpbin.org:443
> > > User-Agent: curl/7.54.0
> > > Proxy-Connection: Keep-Alive
> > >
> >
> > < HTTP/1.1 200 Connection established
> > <
> > * Proxy replied OK to CONNECT request
> > * ALPN, offering h2
> > * ALPN, offering http/1.1
> > * Cipher selection:
> > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > * successfully set certificate verify locations:
> > * CAfile: /etc/ssl/cert.pem
> > CApath: none
> > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > * TLSv1.2 (IN), TLS handshake, Server hello (2):
> > * TLSv1.2 (IN), TLS handshake, Certificate (11):
> > * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> > * TLSv1.2 (IN), TLS handshake, Server finished (14):
> > * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> > * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
> > * TLSv1.2 (OUT), TLS handshake, Finished (20):
> > * TLSv1.2 (IN), TLS change cipher, Client hello (1):
> > * TLSv1.2 (IN), TLS handshake, Finished (20):
> > * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
> > * ALPN, server accepted to use h2
> > * Server certificate:
> > * subject: CN=httpbin.org
> > * start date: Jan 18 00:00:00 2020 GMT
> > * expire date: Feb 18 12:00:00 2021 GMT
> > * subjectAltName: host "httpbin.org" matched cert's "httpbin.org"
> > * issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
> > * SSL certificate verify ok.
> > * Using HTTP2, server supports multi-use
> > * Connection state changed (HTTP/2 confirmed)
> > * Copying HTTP/2 data in stream buffer to connection buffer after
> > upgrade:
> > len=0
> > * Using Stream ID: 1 (easy handle 0x7fa826808c00)
> > > GET /get?answer=42 HTTP/2
> > > Host: httpbin.org
> > > User-Agent: curl/7.54.0
> > > Accept: */*
> > >
> >
> > * Connection state changed (MAX_CONCURRENT_STREAMS updated)!
> > < HTTP/2 200
> > < date: Sat, 05 Dec 2020 06:05:52 GMT
> > < content-type: application/json
> > < content-length: 288
> > < server: gunicorn/19.9.0
> > < access-control-allow-origin: *
> > < access-control-allow-credentials: true
> > <
> > {
> > "args": {
> > "answer": "42"
> > },
> > "headers": {
> > "Accept": "*/*",
> > "Host": "httpbin.org",
> > "User-Agent": "curl/7.54.0",
> > "X-Amzn-Trace-Id": "Root=1-5fcb2340-1bc9172e5a20c5b55ee319c6"
> > },
> > "origin": "*103.253.27.108*",
> > "url": "https://httpbin.org/get?answer=42"
> > }
> > * Connection #0 to host 103.253.27.108 left intact
>
>
>
>
> On Thu, Dec 3, 2020 at 9:34 PM Lei Sun <le...@gmail.com> wrote:
>
> > Guess what, I just realized that all three of us are Yahoo Alums
> > LOL
> >
> > On Thu, Dec 3, 2020 at 9:31 PM Lei Sun <le...@gmail.com> wrote:
> >
> > > This is super helpful! Thank you so much Alan and Susan!
> > >
> > > On Thu, Dec 3, 2020 at 1:36 PM Susan Hinrichs <
> > > shinrich@verizonmedia.com>
> > > wrote:
> > >
> > > > The --resolve option is very helpful for using curl to direct
> > > > requests
> > > > to the proxy to terminate.
> > > >
> > > > curl -k -v --resolve 'httbin.org:4443:127.0.0.1'
> > > > https://httpbin.org:4443/get?answer=4a
> > > >
> > > > Adding the -k assuming you are using a self-signed cert in ATS
> > > > for
> > > > testing. Also assuming your ATS is configured to listen for
> > > > TLS on port
> > > > 4443 in this example.
> > > >
> > > > On Thu, Dec 3, 2020 at 1:29 PM Alan Carroll <
> > > > solidwallofcode@verizonmedia.com> wrote:
> > > >
> > > > > You will need to set up the certificates for ATS in that
> > > > > case. Although
> > > > > it is possible to do this in "records.config", that is (IMHO)
> > > > > deprecated
> > > > > because it has been superceded by "ssl_multicert.config". I
> > > > > would start
> > > > > with that directly, it will be easier.
> > > > >
> > > > > https://docs.trafficserver.apache.org/en/latest/admin-guide/files/ssl_multicert.config.en.html
> > > > > <
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.trafficserver.apache.org_en_latest_admin-2Dguide_files_ssl-5Fmulticert.config.en.html&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=GAqM_xZpxNbVqsR-aGvQBjOG3d33Y2-i4ynL-JkEouY&m=7bubaML3NbE3IHatnbQQrdelwW56A0ptWueuP2dNGGU&s=L4iVe956pejQQnlOZXgP2jUNJ85P-HmFam8gu5eji0U&e=
> > > > > >
> > > > > If there is only a single certificate, you will want to use
> > > > > "dest_ip=*"
> > > > > as the configuration which will use that certificate for all
> > > > > outbound
> > > > > connections.
> > > > > You'll need to use different options to curl to test this, as
> > > > > using
> > > > > "--proxy" with an "https" destination will always bypass TLS
> > > > > on the proxy.
> > > > >
> > > > >
> > > > > On Thu, Dec 3, 2020 at 1:03 PM Lei Sun <le...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Alan,
> > > > > >
> > > > > > Thanks for responding! Yes, I learned about the CONNECT
> > > > > > method.
> > > > > >
> > > > > > I can confirm that for the "https" method, I don't want the
> > > > > > client to
> > > > > > do TLS directly with the server. Instead, I'd like the
> > > > > > trafficserver to
> > > > > > take that request, decrypt it, then pretend to be a client,
> > > > > > and do TLS on
> > > > > > behalf of the client with the upstream.
> > > > > >
> > > > > > Thanks,
> > > > > > Lei
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Dec 3, 2020 at 12:13 PM Alan Carroll <
> > > > > > solidwallofcode@verizonmedia.com> wrote:
> > > > > >
> > > > > > > The first thing to note is "curl --proxy" behaves very
> > > > > > > differently if
> > > > > > > the target URL is "http" vs. "https". In the former case,
> > > > > > > curl will do a
> > > > > > > TCP network connection to the proxy and then sends a GET
> > > > > > > request. In the
> > > > > > > latter case, as you can see from the output, it does a
> > > > > > > TCP network
> > > > > > > connection to the proxy and then sends a CONNECT (not a
> > > > > > > GET) to the proxy.
> > > > > > > After this, curl will do TLS negotiation with the
> > > > > > > upstream, NOT with ATS.
> > > > > > > It is unclear from your description if this is what you
> > > > > > > want.
> > > > > > >
> > > > > > > So, first question - should ATS do TLS negotiation with
> > > > > > > the user
> > > > > > > agent and decrypt the request? Or should it just do a
> > > > > > > blind tunnel pass the
> > > > > > > raw bytes to the upstream so the upstream does the TLS
> > > > > > > with the user agent?
> > > > > > >
> > > > > > > On Wed, Dec 2, 2020 at 5:41 PM Lei Sun <lei.sun@gmail.com
> > > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Kit,
> > > > > > > >
> > > > > > > > I'm trying to set up the trafficserver as an
> > > > > > > > intermediary forward
> > > > > > > > proxy.
> > > > > > > >
> > > > > > > > For example,
> > > > > > > > 1) http client send request to trafficserver.
> > > > > > > > 2) trafficserver then pass this request to the
> > > > > > > > downstream proxy
> > > > > > > > 3) downstream proxy then route this request to the
> > > > > > > > origin site
> > > > > > > > 4) origin site send data back to the downstream proxy
> > > > > > > > 5) downstream proxy send data back to trafficserver
> > > > > > > > 6) trafficserver send data back to the http client.
> > > > > > > >
> > > > > > > > I was able to make the entire request chain work if the
> > > > > > > > origin site
> > > > > > > > serves content directly through HTTP.
> > > > > > > >
> > > > > > > > > curl --proxy *http*://127.0.0.1:8080
> > > > > > > > > <
> > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=
> > > > > > > > > >
> > > > > > > > > *http*://httpbin.org/get?answer=4a
> > > > > > > > > <
> > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=
> > > > > > > > > >
> > > > > > > > > -v
> > > > > > > >
> > > > > > > >
> > > > > > > > However, I ran into issues when I was trying to request
> > > > > > > > for content
> > > > > > > > served from HTTPS.
> > > > > > > >
> > > > > > > > $ curl --proxy *http*://127.0.0.1:8080
> > > > > > > > > <
> > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=
> > > > > > > > > >
> > > > > > > > > *https*://httpbin.org/get?answer=4a
> > > > > > > > > <
> > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=
> > > > > > > > > >
> > > > > > > > > -v
> > > > > > > > > * Trying 127.0.0.1...
> > > > > > > > > * TCP_NODELAY set
> > > > > > > > > * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> > > > > > > > > * Establish HTTP proxy tunnel to httpbin.org:443
> > > > > > > > > <
> > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=
> > > > > > > > > >
> > > > > > > > > > CONNECT httpbin.org:443
> > > > > > > > >
> > > > > > > > > <
> > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=
> > > > > > > > > >
> > > > > > > > > HTTP/1.1
> > > > > > > > > > Host: httpbin.org:443
> > > > > > > > >
> > > > > > > > > <
> > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=
> > > > > > > > > >
> > > > > > > > > > User-Agent: curl/7.54.0
> > > > > > > > > > Proxy-Connection: Keep-Alive
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > < HTTP/1.1 200 OK
> > > > > > > > > < Date: Wed, 02 Dec 2020 20:53:31 GMT
> > > > > > > > > < Proxy-Connection: keep-alive
> > > > > > > > > < Server: ATS/10.0.0
> > > > > > > > > <
> > > > > > > > > * Proxy replied OK to CONNECT request
> > > > > > > > > * ALPN, offering h2
> > > > > > > > > * ALPN, offering http/1.1
> > > > > > > > > * Cipher selection:
> > > > > > > > > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STR
> > > > > > > > > ENGTH
> > > > > > > > > * successfully set certificate verify locations:
> > > > > > > > > * CAfile: /etc/ssl/cert.pem
> > > > > > > > > CApath: none
> > > > > > > > > * *TLSv1.2 (OUT), TLS handshake, Client hello (1):*
> > > > > > > > > * error:1400410B:SSL
> > > > > > > > > routines:CONNECT_CR_SRVR_HELLO:wrong version
> > > > > > > > > number
> > > > > > > > > * stopped the pause stream!
> > > > > > > > > * Closing connection 0
> > > > > > > > > curl: (35) error:1400410B:SSL
> > > > > > > > > routines:CONNECT_CR_SRVR_HELLO:wrong
> > > > > > > > > version number
> > > > > > > > >
> > > > > > > >
> > > > > > > > From the error message, it seems that curl was able to
> > > > > > > > connect to
> > > > > > > > the origin server, and even attempted to send the
> > > > > > > > initial TLS handshake,
> > > > > > > > but somehow the handshake wasn't completed.
> > > > > > > >
> > > > > > > > Here are my questions.
> > > > > > > > 1) What's likely the cause?
> > > > > > > > 2) How can I fix it.
> > > > > > > >
> > > > > > > > Thank you!
> > > > > > > > Lei
> > > > > > > >
> > > > > > > >
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAEBAgAGBQJfy0HFAAoJEMrVQenMmzu+OfgP/3y+NyYrRfkeHAOOvGEPDMwm
GY2HSiRX5XobSM/3cftiYw0Rui+0J2gBCyan9JUunuaBykLLXD3n2YcTEYaeyTBR
X6DmyoaGBJT11a5xKI5TbG5DK/IDdR1Mh3GGZ+9qwgOQYQ87JlocOGxKf7pwPnJb
4j8iuyEyupPa/+/i+uTG20Hi1ZTNLG5xy3ygnr6zK4g7s0GVqfWE2WqkfAf+BFiK
e6diAPs4moopeC9solyrZEyo/cnvu7b0V5UJ9tab3F8ayZwbEssGIsWa7yUc0xjG
oxwu0c6Y8pL5YD7TkvnKWTmW5I7r7XGaf4t3EvdSJvGARiPlbFTmRuMiQ5w+niWv
7pFe5tjs9E2luZa0CPMrYFm1Y6BjXdua3qLmo1/v0sdh7P9qVtNnot2GFWDXFWWt
Ca9LKZQbtDE+Dgq44JqbcBzHMkMJEKmZP5+a5fE5dIx83+FHyPnU3RLdSMkY4p34
1etpvs4jKYa3kTZEdD+h1uaAnK9ZMy7q7k+yVphxKwxDu86Mod96ZDFawi8dP9wJ
ivPnxIvZ2tvZL/QKlr8ITgYf+aiHrsnQ5ciVXnDpGq1c7mQAf76UehyQKE6ycTQb
oHYSyN5Da9cWjXop+4/qUx1lRKHHZaAz0uY8tsIdEXViAqOyvsGcQmH9MiXk+akB
FSAYkptZol7eg/fexHCg
=6mu2
-----END PGP SIGNATURE-----
Re: [E] https issue
Posted by Lei Sun <le...@gmail.com>.
Hi Guys,
I followed the instructions from Alan, and set up the certs.
> dest_ip=* ssl_cert_name=host.pem ssl_key_name=host.key
then I followed the instructions from Susan and tested with --resolve
curl -k -v --resolve 'httbin.org:8443:127.0.0.1'
> https://httpbin.org/get?answer=42 -v
It seems that the ats still lets curl create tls handshake directly with
the origin server, which isn't what I'm looking for.
> $ curl -k -v --resolve 'httbin.org:8443:127.0.0.1'
> https://httpbin.org/get?answer=42 -v
> * Added httbin.org:8443:127.0.0.1 to DNS cache
> * Trying 184.72.216.47...
> * TCP_NODELAY set
> * Connected to httpbin.org (184.72.216.47) port 443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/ssl/cert.pem
> CApath: none
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
I'm trying to do the following, notice the "origin": "*103.253.27.108*",
except that I want to make the request to ATS on *127.0.0.1*, then have the
ATS forward this request to *103.253.27.108*, and return the *https*
response from *103.253.27.108* back to the client. How can we make this
happen with ATS please?
$ curl --proxy http://103.253.27.108:80 *https*://httpbin.org/get?answer=42
> -v
> * Trying 103.253.27.108...
> * TCP_NODELAY set
> * Connected to 103.253.27.108 (103.253.27.108) port 80 (#0)
> * Establish HTTP proxy tunnel to httpbin.org:443
> > CONNECT httpbin.org:443 HTTP/1.1
> > Host: httpbin.org:443
> > User-Agent: curl/7.54.0
> > Proxy-Connection: Keep-Alive
> >
> < HTTP/1.1 200 Connection established
> <
> * Proxy replied OK to CONNECT request
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/ssl/cert.pem
> CApath: none
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> * TLSv1.2 (IN), TLS handshake, Server finished (14):
> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
> * TLSv1.2 (OUT), TLS handshake, Finished (20):
> * TLSv1.2 (IN), TLS change cipher, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Finished (20):
> * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
> * ALPN, server accepted to use h2
> * Server certificate:
> * subject: CN=httpbin.org
> * start date: Jan 18 00:00:00 2020 GMT
> * expire date: Feb 18 12:00:00 2021 GMT
> * subjectAltName: host "httpbin.org" matched cert's "httpbin.org"
> * issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
> * SSL certificate verify ok.
> * Using HTTP2, server supports multi-use
> * Connection state changed (HTTP/2 confirmed)
> * Copying HTTP/2 data in stream buffer to connection buffer after upgrade:
> len=0
> * Using Stream ID: 1 (easy handle 0x7fa826808c00)
> > GET /get?answer=42 HTTP/2
> > Host: httpbin.org
> > User-Agent: curl/7.54.0
> > Accept: */*
> >
> * Connection state changed (MAX_CONCURRENT_STREAMS updated)!
> < HTTP/2 200
> < date: Sat, 05 Dec 2020 06:05:52 GMT
> < content-type: application/json
> < content-length: 288
> < server: gunicorn/19.9.0
> < access-control-allow-origin: *
> < access-control-allow-credentials: true
> <
> {
> "args": {
> "answer": "42"
> },
> "headers": {
> "Accept": "*/*",
> "Host": "httpbin.org",
> "User-Agent": "curl/7.54.0",
> "X-Amzn-Trace-Id": "Root=1-5fcb2340-1bc9172e5a20c5b55ee319c6"
> },
> "origin": "*103.253.27.108*",
> "url": "https://httpbin.org/get?answer=42"
> }
> * Connection #0 to host 103.253.27.108 left intact
On Thu, Dec 3, 2020 at 9:34 PM Lei Sun <le...@gmail.com> wrote:
> Guess what, I just realized that all three of us are Yahoo Alums LOL
>
> On Thu, Dec 3, 2020 at 9:31 PM Lei Sun <le...@gmail.com> wrote:
>
>> This is super helpful! Thank you so much Alan and Susan!
>>
>> On Thu, Dec 3, 2020 at 1:36 PM Susan Hinrichs <sh...@verizonmedia.com>
>> wrote:
>>
>>> The --resolve option is very helpful for using curl to direct requests
>>> to the proxy to terminate.
>>>
>>> curl -k -v --resolve 'httbin.org:4443:127.0.0.1'
>>> https://httpbin.org:4443/get?answer=4a
>>>
>>> Adding the -k assuming you are using a self-signed cert in ATS for
>>> testing. Also assuming your ATS is configured to listen for TLS on port
>>> 4443 in this example.
>>>
>>> On Thu, Dec 3, 2020 at 1:29 PM Alan Carroll <
>>> solidwallofcode@verizonmedia.com> wrote:
>>>
>>>> You will need to set up the certificates for ATS in that case. Although
>>>> it is possible to do this in "records.config", that is (IMHO) deprecated
>>>> because it has been superceded by "ssl_multicert.config". I would start
>>>> with that directly, it will be easier.
>>>>
>>>> https://docs.trafficserver.apache.org/en/latest/admin-guide/files/ssl_multicert.config.en.html
>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.trafficserver.apache.org_en_latest_admin-2Dguide_files_ssl-5Fmulticert.config.en.html&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=GAqM_xZpxNbVqsR-aGvQBjOG3d33Y2-i4ynL-JkEouY&m=7bubaML3NbE3IHatnbQQrdelwW56A0ptWueuP2dNGGU&s=L4iVe956pejQQnlOZXgP2jUNJ85P-HmFam8gu5eji0U&e=>
>>>> If there is only a single certificate, you will want to use "dest_ip=*"
>>>> as the configuration which will use that certificate for all outbound
>>>> connections.
>>>> You'll need to use different options to curl to test this, as using
>>>> "--proxy" with an "https" destination will always bypass TLS on the proxy.
>>>>
>>>>
>>>> On Thu, Dec 3, 2020 at 1:03 PM Lei Sun <le...@gmail.com> wrote:
>>>>
>>>>> Hi Alan,
>>>>>
>>>>> Thanks for responding! Yes, I learned about the CONNECT method.
>>>>>
>>>>> I can confirm that for the "https" method, I don't want the client to
>>>>> do TLS directly with the server. Instead, I'd like the trafficserver to
>>>>> take that request, decrypt it, then pretend to be a client, and do TLS on
>>>>> behalf of the client with the upstream.
>>>>>
>>>>> Thanks,
>>>>> Lei
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Dec 3, 2020 at 12:13 PM Alan Carroll <
>>>>> solidwallofcode@verizonmedia.com> wrote:
>>>>>
>>>>>> The first thing to note is "curl --proxy" behaves very differently if
>>>>>> the target URL is "http" vs. "https". In the former case, curl will do a
>>>>>> TCP network connection to the proxy and then sends a GET request. In the
>>>>>> latter case, as you can see from the output, it does a TCP network
>>>>>> connection to the proxy and then sends a CONNECT (not a GET) to the proxy.
>>>>>> After this, curl will do TLS negotiation with the upstream, NOT with ATS.
>>>>>> It is unclear from your description if this is what you want.
>>>>>>
>>>>>> So, first question - should ATS do TLS negotiation with the user
>>>>>> agent and decrypt the request? Or should it just do a blind tunnel pass the
>>>>>> raw bytes to the upstream so the upstream does the TLS with the user agent?
>>>>>>
>>>>>> On Wed, Dec 2, 2020 at 5:41 PM Lei Sun <le...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Kit,
>>>>>>>
>>>>>>> I'm trying to set up the trafficserver as an intermediary forward
>>>>>>> proxy.
>>>>>>>
>>>>>>> For example,
>>>>>>> 1) http client send request to trafficserver.
>>>>>>> 2) trafficserver then pass this request to the downstream proxy
>>>>>>> 3) downstream proxy then route this request to the origin site
>>>>>>> 4) origin site send data back to the downstream proxy
>>>>>>> 5) downstream proxy send data back to trafficserver
>>>>>>> 6) trafficserver send data back to the http client.
>>>>>>>
>>>>>>> I was able to make the entire request chain work if the origin site
>>>>>>> serves content directly through HTTP.
>>>>>>>
>>>>>>>> curl --proxy *http*://127.0.0.1:8080
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>>>>>> *http*://httpbin.org/get?answer=4a
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>>>>>> -v
>>>>>>>
>>>>>>>
>>>>>>> However, I ran into issues when I was trying to request for content
>>>>>>> served from HTTPS.
>>>>>>>
>>>>>>> $ curl --proxy *http*://127.0.0.1:8080
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>>>>>> *https*://httpbin.org/get?answer=4a
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>>>>>> -v
>>>>>>>> * Trying 127.0.0.1...
>>>>>>>> * TCP_NODELAY set
>>>>>>>> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
>>>>>>>> * Establish HTTP proxy tunnel to httpbin.org:443
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>>>> > CONNECT httpbin.org:443
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>>>> HTTP/1.1
>>>>>>>> > Host: httpbin.org:443
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>>>> > User-Agent: curl/7.54.0
>>>>>>>> > Proxy-Connection: Keep-Alive
>>>>>>>> >
>>>>>>>> < HTTP/1.1 200 OK
>>>>>>>> < Date: Wed, 02 Dec 2020 20:53:31 GMT
>>>>>>>> < Proxy-Connection: keep-alive
>>>>>>>> < Server: ATS/10.0.0
>>>>>>>> <
>>>>>>>> * Proxy replied OK to CONNECT request
>>>>>>>> * ALPN, offering h2
>>>>>>>> * ALPN, offering http/1.1
>>>>>>>> * Cipher selection:
>>>>>>>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>>>>>>>> * successfully set certificate verify locations:
>>>>>>>> * CAfile: /etc/ssl/cert.pem
>>>>>>>> CApath: none
>>>>>>>> * *TLSv1.2 (OUT), TLS handshake, Client hello (1):*
>>>>>>>> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version
>>>>>>>> number
>>>>>>>> * stopped the pause stream!
>>>>>>>> * Closing connection 0
>>>>>>>> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
>>>>>>>> version number
>>>>>>>>
>>>>>>>
>>>>>>> From the error message, it seems that curl was able to connect to
>>>>>>> the origin server, and even attempted to send the initial TLS handshake,
>>>>>>> but somehow the handshake wasn't completed.
>>>>>>>
>>>>>>> Here are my questions.
>>>>>>> 1) What's likely the cause?
>>>>>>> 2) How can I fix it.
>>>>>>>
>>>>>>> Thank you!
>>>>>>> Lei
>>>>>>>
>>>>>>>
--
Stay Hungry, Stay Foolish.
Lei Sun
Cell: 408-306-9199
Re: [E] https issue
Posted by Lei Sun <le...@gmail.com>.
Guess what, I just realized that all three of us are Yahoo Alums LOL
On Thu, Dec 3, 2020 at 9:31 PM Lei Sun <le...@gmail.com> wrote:
> This is super helpful! Thank you so much Alan and Susan!
>
> On Thu, Dec 3, 2020 at 1:36 PM Susan Hinrichs <sh...@verizonmedia.com>
> wrote:
>
>> The --resolve option is very helpful for using curl to direct requests to
>> the proxy to terminate.
>>
>> curl -k -v --resolve 'httbin.org:4443:127.0.0.1'
>> https://httpbin.org:4443/get?answer=4a
>>
>> Adding the -k assuming you are using a self-signed cert in ATS for
>> testing. Also assuming your ATS is configured to listen for TLS on port
>> 4443 in this example.
>>
>> On Thu, Dec 3, 2020 at 1:29 PM Alan Carroll <
>> solidwallofcode@verizonmedia.com> wrote:
>>
>>> You will need to set up the certificates for ATS in that case. Although
>>> it is possible to do this in "records.config", that is (IMHO) deprecated
>>> because it has been superceded by "ssl_multicert.config". I would start
>>> with that directly, it will be easier.
>>>
>>> https://docs.trafficserver.apache.org/en/latest/admin-guide/files/ssl_multicert.config.en.html
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.trafficserver.apache.org_en_latest_admin-2Dguide_files_ssl-5Fmulticert.config.en.html&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=GAqM_xZpxNbVqsR-aGvQBjOG3d33Y2-i4ynL-JkEouY&m=7bubaML3NbE3IHatnbQQrdelwW56A0ptWueuP2dNGGU&s=L4iVe956pejQQnlOZXgP2jUNJ85P-HmFam8gu5eji0U&e=>
>>> If there is only a single certificate, you will want to use "dest_ip=*"
>>> as the configuration which will use that certificate for all outbound
>>> connections.
>>> You'll need to use different options to curl to test this, as using
>>> "--proxy" with an "https" destination will always bypass TLS on the proxy.
>>>
>>>
>>> On Thu, Dec 3, 2020 at 1:03 PM Lei Sun <le...@gmail.com> wrote:
>>>
>>>> Hi Alan,
>>>>
>>>> Thanks for responding! Yes, I learned about the CONNECT method.
>>>>
>>>> I can confirm that for the "https" method, I don't want the client to
>>>> do TLS directly with the server. Instead, I'd like the trafficserver to
>>>> take that request, decrypt it, then pretend to be a client, and do TLS on
>>>> behalf of the client with the upstream.
>>>>
>>>> Thanks,
>>>> Lei
>>>>
>>>>
>>>>
>>>> On Thu, Dec 3, 2020 at 12:13 PM Alan Carroll <
>>>> solidwallofcode@verizonmedia.com> wrote:
>>>>
>>>>> The first thing to note is "curl --proxy" behaves very differently if
>>>>> the target URL is "http" vs. "https". In the former case, curl will do a
>>>>> TCP network connection to the proxy and then sends a GET request. In the
>>>>> latter case, as you can see from the output, it does a TCP network
>>>>> connection to the proxy and then sends a CONNECT (not a GET) to the proxy.
>>>>> After this, curl will do TLS negotiation with the upstream, NOT with ATS.
>>>>> It is unclear from your description if this is what you want.
>>>>>
>>>>> So, first question - should ATS do TLS negotiation with the user agent
>>>>> and decrypt the request? Or should it just do a blind tunnel pass the raw
>>>>> bytes to the upstream so the upstream does the TLS with the user agent?
>>>>>
>>>>> On Wed, Dec 2, 2020 at 5:41 PM Lei Sun <le...@gmail.com> wrote:
>>>>>
>>>>>> Hi Kit,
>>>>>>
>>>>>> I'm trying to set up the trafficserver as an intermediary forward
>>>>>> proxy.
>>>>>>
>>>>>> For example,
>>>>>> 1) http client send request to trafficserver.
>>>>>> 2) trafficserver then pass this request to the downstream proxy
>>>>>> 3) downstream proxy then route this request to the origin site
>>>>>> 4) origin site send data back to the downstream proxy
>>>>>> 5) downstream proxy send data back to trafficserver
>>>>>> 6) trafficserver send data back to the http client.
>>>>>>
>>>>>> I was able to make the entire request chain work if the origin site
>>>>>> serves content directly through HTTP.
>>>>>>
>>>>>>> curl --proxy *http*://127.0.0.1:8080
>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>>>>> *http*://httpbin.org/get?answer=4a
>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>>>>> -v
>>>>>>
>>>>>>
>>>>>> However, I ran into issues when I was trying to request for content
>>>>>> served from HTTPS.
>>>>>>
>>>>>> $ curl --proxy *http*://127.0.0.1:8080
>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>>>>> *https*://httpbin.org/get?answer=4a
>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>>>>> -v
>>>>>>> * Trying 127.0.0.1...
>>>>>>> * TCP_NODELAY set
>>>>>>> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
>>>>>>> * Establish HTTP proxy tunnel to httpbin.org:443
>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>>> > CONNECT httpbin.org:443
>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>>> HTTP/1.1
>>>>>>> > Host: httpbin.org:443
>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>>> > User-Agent: curl/7.54.0
>>>>>>> > Proxy-Connection: Keep-Alive
>>>>>>> >
>>>>>>> < HTTP/1.1 200 OK
>>>>>>> < Date: Wed, 02 Dec 2020 20:53:31 GMT
>>>>>>> < Proxy-Connection: keep-alive
>>>>>>> < Server: ATS/10.0.0
>>>>>>> <
>>>>>>> * Proxy replied OK to CONNECT request
>>>>>>> * ALPN, offering h2
>>>>>>> * ALPN, offering http/1.1
>>>>>>> * Cipher selection:
>>>>>>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>>>>>>> * successfully set certificate verify locations:
>>>>>>> * CAfile: /etc/ssl/cert.pem
>>>>>>> CApath: none
>>>>>>> * *TLSv1.2 (OUT), TLS handshake, Client hello (1):*
>>>>>>> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version
>>>>>>> number
>>>>>>> * stopped the pause stream!
>>>>>>> * Closing connection 0
>>>>>>> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
>>>>>>> version number
>>>>>>>
>>>>>>
>>>>>> From the error message, it seems that curl was able to connect to the
>>>>>> origin server, and even attempted to send the initial TLS handshake, but
>>>>>> somehow the handshake wasn't completed.
>>>>>>
>>>>>> Here are my questions.
>>>>>> 1) What's likely the cause?
>>>>>> 2) How can I fix it.
>>>>>>
>>>>>> Thank you!
>>>>>> Lei
>>>>>>
>>>>>>
Re: [E] https issue
Posted by Lei Sun <le...@gmail.com>.
This is super helpful! Thank you so much Alan and Susan!
On Thu, Dec 3, 2020 at 1:36 PM Susan Hinrichs <sh...@verizonmedia.com>
wrote:
> The --resolve option is very helpful for using curl to direct requests to
> the proxy to terminate.
>
> curl -k -v --resolve 'httbin.org:4443:127.0.0.1'
> https://httpbin.org:4443/get?answer=4a
>
> Adding the -k assuming you are using a self-signed cert in ATS for
> testing. Also assuming your ATS is configured to listen for TLS on port
> 4443 in this example.
>
> On Thu, Dec 3, 2020 at 1:29 PM Alan Carroll <
> solidwallofcode@verizonmedia.com> wrote:
>
>> You will need to set up the certificates for ATS in that case. Although
>> it is possible to do this in "records.config", that is (IMHO) deprecated
>> because it has been superceded by "ssl_multicert.config". I would start
>> with that directly, it will be easier.
>>
>> https://docs.trafficserver.apache.org/en/latest/admin-guide/files/ssl_multicert.config.en.html
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.trafficserver.apache.org_en_latest_admin-2Dguide_files_ssl-5Fmulticert.config.en.html&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=GAqM_xZpxNbVqsR-aGvQBjOG3d33Y2-i4ynL-JkEouY&m=7bubaML3NbE3IHatnbQQrdelwW56A0ptWueuP2dNGGU&s=L4iVe956pejQQnlOZXgP2jUNJ85P-HmFam8gu5eji0U&e=>
>> If there is only a single certificate, you will want to use "dest_ip=*"
>> as the configuration which will use that certificate for all outbound
>> connections.
>> You'll need to use different options to curl to test this, as using
>> "--proxy" with an "https" destination will always bypass TLS on the proxy.
>>
>>
>> On Thu, Dec 3, 2020 at 1:03 PM Lei Sun <le...@gmail.com> wrote:
>>
>>> Hi Alan,
>>>
>>> Thanks for responding! Yes, I learned about the CONNECT method.
>>>
>>> I can confirm that for the "https" method, I don't want the client to do
>>> TLS directly with the server. Instead, I'd like the trafficserver to take
>>> that request, decrypt it, then pretend to be a client, and do TLS on
>>> behalf of the client with the upstream.
>>>
>>> Thanks,
>>> Lei
>>>
>>>
>>>
>>> On Thu, Dec 3, 2020 at 12:13 PM Alan Carroll <
>>> solidwallofcode@verizonmedia.com> wrote:
>>>
>>>> The first thing to note is "curl --proxy" behaves very differently if
>>>> the target URL is "http" vs. "https". In the former case, curl will do a
>>>> TCP network connection to the proxy and then sends a GET request. In the
>>>> latter case, as you can see from the output, it does a TCP network
>>>> connection to the proxy and then sends a CONNECT (not a GET) to the proxy.
>>>> After this, curl will do TLS negotiation with the upstream, NOT with ATS.
>>>> It is unclear from your description if this is what you want.
>>>>
>>>> So, first question - should ATS do TLS negotiation with the user agent
>>>> and decrypt the request? Or should it just do a blind tunnel pass the raw
>>>> bytes to the upstream so the upstream does the TLS with the user agent?
>>>>
>>>> On Wed, Dec 2, 2020 at 5:41 PM Lei Sun <le...@gmail.com> wrote:
>>>>
>>>>> Hi Kit,
>>>>>
>>>>> I'm trying to set up the trafficserver as an intermediary forward
>>>>> proxy.
>>>>>
>>>>> For example,
>>>>> 1) http client send request to trafficserver.
>>>>> 2) trafficserver then pass this request to the downstream proxy
>>>>> 3) downstream proxy then route this request to the origin site
>>>>> 4) origin site send data back to the downstream proxy
>>>>> 5) downstream proxy send data back to trafficserver
>>>>> 6) trafficserver send data back to the http client.
>>>>>
>>>>> I was able to make the entire request chain work if the origin site
>>>>> serves content directly through HTTP.
>>>>>
>>>>>> curl --proxy *http*://127.0.0.1:8080
>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>>>> *http*://httpbin.org/get?answer=4a
>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>>>> -v
>>>>>
>>>>>
>>>>> However, I ran into issues when I was trying to request for content
>>>>> served from HTTPS.
>>>>>
>>>>> $ curl --proxy *http*://127.0.0.1:8080
>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>>>> *https*://httpbin.org/get?answer=4a
>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>>>> -v
>>>>>> * Trying 127.0.0.1...
>>>>>> * TCP_NODELAY set
>>>>>> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
>>>>>> * Establish HTTP proxy tunnel to httpbin.org:443
>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>> > CONNECT httpbin.org:443
>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>> HTTP/1.1
>>>>>> > Host: httpbin.org:443
>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>> > User-Agent: curl/7.54.0
>>>>>> > Proxy-Connection: Keep-Alive
>>>>>> >
>>>>>> < HTTP/1.1 200 OK
>>>>>> < Date: Wed, 02 Dec 2020 20:53:31 GMT
>>>>>> < Proxy-Connection: keep-alive
>>>>>> < Server: ATS/10.0.0
>>>>>> <
>>>>>> * Proxy replied OK to CONNECT request
>>>>>> * ALPN, offering h2
>>>>>> * ALPN, offering http/1.1
>>>>>> * Cipher selection:
>>>>>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>>>>>> * successfully set certificate verify locations:
>>>>>> * CAfile: /etc/ssl/cert.pem
>>>>>> CApath: none
>>>>>> * *TLSv1.2 (OUT), TLS handshake, Client hello (1):*
>>>>>> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version
>>>>>> number
>>>>>> * stopped the pause stream!
>>>>>> * Closing connection 0
>>>>>> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
>>>>>> version number
>>>>>>
>>>>>
>>>>> From the error message, it seems that curl was able to connect to the
>>>>> origin server, and even attempted to send the initial TLS handshake, but
>>>>> somehow the handshake wasn't completed.
>>>>>
>>>>> Here are my questions.
>>>>> 1) What's likely the cause?
>>>>> 2) How can I fix it.
>>>>>
>>>>> Thank you!
>>>>> Lei
>>>>>
>>>>>
--
Stay Hungry, Stay Foolish.
Lei Sun
Cell: 408-306-9199
Re: [E] https issue
Posted by Susan Hinrichs <sh...@verizonmedia.com>.
The --resolve option is very helpful for using curl to direct requests to
the proxy to terminate.
curl -k -v --resolve 'httbin.org:4443:127.0.0.1'
https://httpbin.org:4443/get?answer=4a
Adding the -k assuming you are using a self-signed cert in ATS for
testing. Also assuming your ATS is configured to listen for TLS on port
4443 in this example.
On Thu, Dec 3, 2020 at 1:29 PM Alan Carroll <
solidwallofcode@verizonmedia.com> wrote:
> You will need to set up the certificates for ATS in that case. Although it
> is possible to do this in "records.config", that is (IMHO) deprecated
> because it has been superceded by "ssl_multicert.config". I would start
> with that directly, it will be easier.
>
> https://docs.trafficserver.apache.org/en/latest/admin-guide/files/ssl_multicert.config.en.html
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.trafficserver.apache.org_en_latest_admin-2Dguide_files_ssl-5Fmulticert.config.en.html&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=GAqM_xZpxNbVqsR-aGvQBjOG3d33Y2-i4ynL-JkEouY&m=7bubaML3NbE3IHatnbQQrdelwW56A0ptWueuP2dNGGU&s=L4iVe956pejQQnlOZXgP2jUNJ85P-HmFam8gu5eji0U&e=>
> If there is only a single certificate, you will want to use "dest_ip=*" as
> the configuration which will use that certificate for all outbound
> connections.
> You'll need to use different options to curl to test this, as using
> "--proxy" with an "https" destination will always bypass TLS on the proxy.
>
>
> On Thu, Dec 3, 2020 at 1:03 PM Lei Sun <le...@gmail.com> wrote:
>
>> Hi Alan,
>>
>> Thanks for responding! Yes, I learned about the CONNECT method.
>>
>> I can confirm that for the "https" method, I don't want the client to do
>> TLS directly with the server. Instead, I'd like the trafficserver to take
>> that request, decrypt it, then pretend to be a client, and do TLS on
>> behalf of the client with the upstream.
>>
>> Thanks,
>> Lei
>>
>>
>>
>> On Thu, Dec 3, 2020 at 12:13 PM Alan Carroll <
>> solidwallofcode@verizonmedia.com> wrote:
>>
>>> The first thing to note is "curl --proxy" behaves very differently if
>>> the target URL is "http" vs. "https". In the former case, curl will do a
>>> TCP network connection to the proxy and then sends a GET request. In the
>>> latter case, as you can see from the output, it does a TCP network
>>> connection to the proxy and then sends a CONNECT (not a GET) to the proxy.
>>> After this, curl will do TLS negotiation with the upstream, NOT with ATS.
>>> It is unclear from your description if this is what you want.
>>>
>>> So, first question - should ATS do TLS negotiation with the user agent
>>> and decrypt the request? Or should it just do a blind tunnel pass the raw
>>> bytes to the upstream so the upstream does the TLS with the user agent?
>>>
>>> On Wed, Dec 2, 2020 at 5:41 PM Lei Sun <le...@gmail.com> wrote:
>>>
>>>> Hi Kit,
>>>>
>>>> I'm trying to set up the trafficserver as an intermediary forward proxy.
>>>>
>>>> For example,
>>>> 1) http client send request to trafficserver.
>>>> 2) trafficserver then pass this request to the downstream proxy
>>>> 3) downstream proxy then route this request to the origin site
>>>> 4) origin site send data back to the downstream proxy
>>>> 5) downstream proxy send data back to trafficserver
>>>> 6) trafficserver send data back to the http client.
>>>>
>>>> I was able to make the entire request chain work if the origin site
>>>> serves content directly through HTTP.
>>>>
>>>>> curl --proxy *http*://127.0.0.1:8080
>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>>> *http*://httpbin.org/get?answer=4a
>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>>> -v
>>>>
>>>>
>>>> However, I ran into issues when I was trying to request for content
>>>> served from HTTPS.
>>>>
>>>> $ curl --proxy *http*://127.0.0.1:8080
>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>>> *https*://httpbin.org/get?answer=4a
>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>>> -v
>>>>> * Trying 127.0.0.1...
>>>>> * TCP_NODELAY set
>>>>> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
>>>>> * Establish HTTP proxy tunnel to httpbin.org:443
>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>> > CONNECT httpbin.org:443
>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>> HTTP/1.1
>>>>> > Host: httpbin.org:443
>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>> > User-Agent: curl/7.54.0
>>>>> > Proxy-Connection: Keep-Alive
>>>>> >
>>>>> < HTTP/1.1 200 OK
>>>>> < Date: Wed, 02 Dec 2020 20:53:31 GMT
>>>>> < Proxy-Connection: keep-alive
>>>>> < Server: ATS/10.0.0
>>>>> <
>>>>> * Proxy replied OK to CONNECT request
>>>>> * ALPN, offering h2
>>>>> * ALPN, offering http/1.1
>>>>> * Cipher selection:
>>>>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>>>>> * successfully set certificate verify locations:
>>>>> * CAfile: /etc/ssl/cert.pem
>>>>> CApath: none
>>>>> * *TLSv1.2 (OUT), TLS handshake, Client hello (1):*
>>>>> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version
>>>>> number
>>>>> * stopped the pause stream!
>>>>> * Closing connection 0
>>>>> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
>>>>> version number
>>>>>
>>>>
>>>> From the error message, it seems that curl was able to connect to the
>>>> origin server, and even attempted to send the initial TLS handshake, but
>>>> somehow the handshake wasn't completed.
>>>>
>>>> Here are my questions.
>>>> 1) What's likely the cause?
>>>> 2) How can I fix it.
>>>>
>>>> Thank you!
>>>> Lei
>>>>
>>>>
Re: [E] https issue
Posted by Alan Carroll <so...@verizonmedia.com>.
You will need to set up the certificates for ATS in that case. Although it
is possible to do this in "records.config", that is (IMHO) deprecated
because it has been superceded by "ssl_multicert.config". I would start
with that directly, it will be easier.
https://docs.trafficserver.apache.org/en/latest/admin-guide/files/ssl_multicert.config.en.html
If there is only a single certificate, you will want to use "dest_ip=*" as
the configuration which will use that certificate for all outbound
connections.
You'll need to use different options to curl to test this, as using
"--proxy" with an "https" destination will always bypass TLS on the proxy.
On Thu, Dec 3, 2020 at 1:03 PM Lei Sun <le...@gmail.com> wrote:
> Hi Alan,
>
> Thanks for responding! Yes, I learned about the CONNECT method.
>
> I can confirm that for the "https" method, I don't want the client to do
> TLS directly with the server. Instead, I'd like the trafficserver to take
> that request, decrypt it, then pretend to be a client, and do TLS on
> behalf of the client with the upstream.
>
> Thanks,
> Lei
>
>
>
> On Thu, Dec 3, 2020 at 12:13 PM Alan Carroll <
> solidwallofcode@verizonmedia.com> wrote:
>
>> The first thing to note is "curl --proxy" behaves very differently if the
>> target URL is "http" vs. "https". In the former case, curl will do a TCP
>> network connection to the proxy and then sends a GET request. In the latter
>> case, as you can see from the output, it does a TCP network connection to
>> the proxy and then sends a CONNECT (not a GET) to the proxy. After this,
>> curl will do TLS negotiation with the upstream, NOT with ATS. It is unclear
>> from your description if this is what you want.
>>
>> So, first question - should ATS do TLS negotiation with the user agent
>> and decrypt the request? Or should it just do a blind tunnel pass the raw
>> bytes to the upstream so the upstream does the TLS with the user agent?
>>
>> On Wed, Dec 2, 2020 at 5:41 PM Lei Sun <le...@gmail.com> wrote:
>>
>>> Hi Kit,
>>>
>>> I'm trying to set up the trafficserver as an intermediary forward proxy.
>>>
>>> For example,
>>> 1) http client send request to trafficserver.
>>> 2) trafficserver then pass this request to the downstream proxy
>>> 3) downstream proxy then route this request to the origin site
>>> 4) origin site send data back to the downstream proxy
>>> 5) downstream proxy send data back to trafficserver
>>> 6) trafficserver send data back to the http client.
>>>
>>> I was able to make the entire request chain work if the origin site
>>> serves content directly through HTTP.
>>>
>>>> curl --proxy *http*://127.0.0.1:8080
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>> *http*://httpbin.org/get?answer=4a
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>> -v
>>>
>>>
>>> However, I ran into issues when I was trying to request for content
>>> served from HTTPS.
>>>
>>> $ curl --proxy *http*://127.0.0.1:8080
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>> *https*://httpbin.org/get?answer=4a
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>> -v
>>>> * Trying 127.0.0.1...
>>>> * TCP_NODELAY set
>>>> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
>>>> * Establish HTTP proxy tunnel to httpbin.org:443
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>> > CONNECT httpbin.org:443
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>> HTTP/1.1
>>>> > Host: httpbin.org:443
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>> > User-Agent: curl/7.54.0
>>>> > Proxy-Connection: Keep-Alive
>>>> >
>>>> < HTTP/1.1 200 OK
>>>> < Date: Wed, 02 Dec 2020 20:53:31 GMT
>>>> < Proxy-Connection: keep-alive
>>>> < Server: ATS/10.0.0
>>>> <
>>>> * Proxy replied OK to CONNECT request
>>>> * ALPN, offering h2
>>>> * ALPN, offering http/1.1
>>>> * Cipher selection:
>>>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>>>> * successfully set certificate verify locations:
>>>> * CAfile: /etc/ssl/cert.pem
>>>> CApath: none
>>>> * *TLSv1.2 (OUT), TLS handshake, Client hello (1):*
>>>> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
>>>> * stopped the pause stream!
>>>> * Closing connection 0
>>>> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
>>>> version number
>>>>
>>>
>>> From the error message, it seems that curl was able to connect to the
>>> origin server, and even attempted to send the initial TLS handshake, but
>>> somehow the handshake wasn't completed.
>>>
>>> Here are my questions.
>>> 1) What's likely the cause?
>>> 2) How can I fix it.
>>>
>>> Thank you!
>>> Lei
>>>
>>>
Re: [E] https issue
Posted by Lei Sun <le...@gmail.com>.
Hi Alan,
Thanks for responding! Yes, I learned about the CONNECT method.
I can confirm that for the "https" method, I don't want the client to do
TLS directly with the server. Instead, I'd like the trafficserver to take
that request, decrypt it, then pretend to be a client, and do TLS on
behalf of the client with the upstream.
Thanks,
Lei
On Thu, Dec 3, 2020 at 12:13 PM Alan Carroll <
solidwallofcode@verizonmedia.com> wrote:
> The first thing to note is "curl --proxy" behaves very differently if the
> target URL is "http" vs. "https". In the former case, curl will do a TCP
> network connection to the proxy and then sends a GET request. In the latter
> case, as you can see from the output, it does a TCP network connection to
> the proxy and then sends a CONNECT (not a GET) to the proxy. After this,
> curl will do TLS negotiation with the upstream, NOT with ATS. It is unclear
> from your description if this is what you want.
>
> So, first question - should ATS do TLS negotiation with the user agent and
> decrypt the request? Or should it just do a blind tunnel pass the raw bytes
> to the upstream so the upstream does the TLS with the user agent?
>
> On Wed, Dec 2, 2020 at 5:41 PM Lei Sun <le...@gmail.com> wrote:
>
>> Hi Kit,
>>
>> I'm trying to set up the trafficserver as an intermediary forward proxy.
>>
>> For example,
>> 1) http client send request to trafficserver.
>> 2) trafficserver then pass this request to the downstream proxy
>> 3) downstream proxy then route this request to the origin site
>> 4) origin site send data back to the downstream proxy
>> 5) downstream proxy send data back to trafficserver
>> 6) trafficserver send data back to the http client.
>>
>> I was able to make the entire request chain work if the origin site
>> serves content directly through HTTP.
>>
>>> curl --proxy *http*://127.0.0.1:8080
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>> *http*://httpbin.org/get?answer=4a
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>> -v
>>
>>
>> However, I ran into issues when I was trying to request for content
>> served from HTTPS.
>>
>> $ curl --proxy *http*://127.0.0.1:8080
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>> *https*://httpbin.org/get?answer=4a
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>> -v
>>> * Trying 127.0.0.1...
>>> * TCP_NODELAY set
>>> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
>>> * Establish HTTP proxy tunnel to httpbin.org:443
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>> > CONNECT httpbin.org:443
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>> HTTP/1.1
>>> > Host: httpbin.org:443
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>> > User-Agent: curl/7.54.0
>>> > Proxy-Connection: Keep-Alive
>>> >
>>> < HTTP/1.1 200 OK
>>> < Date: Wed, 02 Dec 2020 20:53:31 GMT
>>> < Proxy-Connection: keep-alive
>>> < Server: ATS/10.0.0
>>> <
>>> * Proxy replied OK to CONNECT request
>>> * ALPN, offering h2
>>> * ALPN, offering http/1.1
>>> * Cipher selection:
>>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>>> * successfully set certificate verify locations:
>>> * CAfile: /etc/ssl/cert.pem
>>> CApath: none
>>> * *TLSv1.2 (OUT), TLS handshake, Client hello (1):*
>>> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
>>> * stopped the pause stream!
>>> * Closing connection 0
>>> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
>>> version number
>>>
>>
>> From the error message, it seems that curl was able to connect to the
>> origin server, and even attempted to send the initial TLS handshake, but
>> somehow the handshake wasn't completed.
>>
>> Here are my questions.
>> 1) What's likely the cause?
>> 2) How can I fix it.
>>
>> Thank you!
>> Lei
>>
>>
Re: [E] https issue
Posted by Alan Carroll <so...@verizonmedia.com>.
The first thing to note is "curl --proxy" behaves very differently if the
target URL is "http" vs. "https". In the former case, curl will do a TCP
network connection to the proxy and then sends a GET request. In the latter
case, as you can see from the output, it does a TCP network connection to
the proxy and then sends a CONNECT (not a GET) to the proxy. After this,
curl will do TLS negotiation with the upstream, NOT with ATS. It is unclear
from your description if this is what you want.
So, first question - should ATS do TLS negotiation with the user agent and
decrypt the request? Or should it just do a blind tunnel pass the raw bytes
to the upstream so the upstream does the TLS with the user agent?
On Wed, Dec 2, 2020 at 5:41 PM Lei Sun <le...@gmail.com> wrote:
> Hi Kit,
>
> I'm trying to set up the trafficserver as an intermediary forward proxy.
>
> For example,
> 1) http client send request to trafficserver.
> 2) trafficserver then pass this request to the downstream proxy
> 3) downstream proxy then route this request to the origin site
> 4) origin site send data back to the downstream proxy
> 5) downstream proxy send data back to trafficserver
> 6) trafficserver send data back to the http client.
>
> I was able to make the entire request chain work if the origin site serves
> content directly through HTTP.
>
>> curl --proxy *http*://127.0.0.1:8080
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>> *http*://httpbin.org/get?answer=4a
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>> -v
>
>
> However, I ran into issues when I was trying to request for content served
> from HTTPS.
>
> $ curl --proxy *http*://127.0.0.1:8080
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>> *https*://httpbin.org/get?answer=4a
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>> -v
>> * Trying 127.0.0.1...
>> * TCP_NODELAY set
>> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
>> * Establish HTTP proxy tunnel to httpbin.org:443
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>> > CONNECT httpbin.org:443
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>> HTTP/1.1
>> > Host: httpbin.org:443
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>> > User-Agent: curl/7.54.0
>> > Proxy-Connection: Keep-Alive
>> >
>> < HTTP/1.1 200 OK
>> < Date: Wed, 02 Dec 2020 20:53:31 GMT
>> < Proxy-Connection: keep-alive
>> < Server: ATS/10.0.0
>> <
>> * Proxy replied OK to CONNECT request
>> * ALPN, offering h2
>> * ALPN, offering http/1.1
>> * Cipher selection:
>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>> * successfully set certificate verify locations:
>> * CAfile: /etc/ssl/cert.pem
>> CApath: none
>> * *TLSv1.2 (OUT), TLS handshake, Client hello (1):*
>> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
>> * stopped the pause stream!
>> * Closing connection 0
>> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
>> version number
>>
>
> From the error message, it seems that curl was able to connect to the
> origin server, and even attempted to send the initial TLS handshake, but
> somehow the handshake wasn't completed.
>
> Here are my questions.
> 1) What's likely the cause?
> 2) How can I fix it.
>
> Thank you!
> Lei
>
>