You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Krishna Sankaran <kr...@yahoo.com.INVALID> on 2019/06/21 00:04:17 UTC
HTTP headers for Redirected HEAD requests
I am using the following java snippet for a HEAD request. The request gets redirected couple of times, a 301 followed by 302 and dinally gets a 200.When i use the following code snippet, i get the status as 200 OK, however all the header fields are from the 1st response (301). The Content-Length header is present in the last response, but i don't see that in the header's received by the code. Is there a knob to get the last header or all headers?
private static final String SAMPLE_URL = "https://www.dropbox.com/s/<something>/test_out4.mp4";
public static void main(String[] args) throws IOException, URISyntaxException {
HttpClient client = HttpClientBuilder.create().build();
HttpHead request = new HttpHead(new URI(SAMPLE_URL));
HttpResponse response = client.execute(request);
System.out.println(response.getStatusLine());
for (Header header : response.getAllHeaders()) {
System.out.println(header.getName() + ": " + header.getValue());
}
}
It returns the code as 200 OK, but the Content-Type is incorrect, even
the
HTTP/1.1 200 OK <<< Status is 200 Server: nginx Date: Thu, 20 Jun 2019 02:22:58 GMT Content-Type: text/html; charset=utf-8 << Content type is char <<<< Content-Length is missing!!!!!
curl -I https://www.dropbox.com/s//test_out4.mp4 << Curl works correctly
HTTP/1.1 301 Moved Permanently <<< Status 301 Server: nginx
Date: Thu, 20 Jun 2019 02:20:50 GMT Content-Type: text/html; charset=utf-8 <<< Content type text Connection: keep-alive
....
HTTP/1.1 302 Found << second redirect Server: nginx
Date: Thu, 20 Jun 2019 02:36:03 GMT
Content-Type: text/html; charset=utf-8
....
HTTP/1.1 200 OK <<< Status finally 200 Server: nginx
Date: Thu, 20 Jun 2019 02:36:04 GMT
Content-Type: video/mp4 << content type correct
Content-Length: 92894175 << length correct Connection: keep-alive
Re: HTTP headers for Redirected HEAD requests
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2019-06-23 at 13:32 +0000, Krishna Sankaran wrote:
> Attached the curl output for the same URL.
> There is a 301, followed by 302 followed by 200.
> Content-Type: text/html; charset=utf-8 is seen for the 301 and 302.
> and there is no Content-Length
> But finally, for the 200 i see the correct Content-Type and Length.
> Content-Type: video/mp4 <<<<
> Content-Length: 92894175 <<<<<
> I set the user agent to "some other string" to see if the server is
> doing something strange based on UserAgent.
>
I am not sure what difference that makes. The server responds
differently to requests generated by curl.
Oleg
>
> curl -L -I https://www.dropbox.com/s/<sommedir>/test_out4.mp4
> HTTP/1.1 301 Moved Permanently
> Server: nginx
> Date: Sun, 23 Jun 2019 13:25:44 GMT
> Content-Type: text/html; charset=utf-8
> Connection: keep-alive
> Cache-Control: no-cache
> Content-Security-Policy: sandbox
> Location: /s/raw/<somedir>/test_out4.mp4
> Pragma: no-cache
> Referrer-Policy: origin-when-cross-origin
> Set-Cookie: locale=en; Domain=dropbox.com; expires=Fri, 21 Jun 2024
> 13:25:44 GMT; Path=/; secure
> Set-Cookie: gvc=MTA3NDcwOTg3MzE2NzIyNTE1MjMxNTg0OTM2OTA4NDcyMDExNzcx;
> expires=Fri, 21 Jun 2024 13:25:44 GMT; httponly; Path=/; secure
> Set-Cookie: flash=; Domain=dropbox.com; expires=Sun, 23 Jun 2019
> 13:25:44 GMT; Path=/; secure
> Set-Cookie: puc=; expires=Sun, 23 Jun 2019 13:25:44 GMT; httponly;
> Path=/; secure
> Set-Cookie: bang=; Domain=dropbox.com; expires=Sun, 23 Jun 2019
> 13:25:44 GMT; Path=/; secure
> Set-Cookie: t=dFl-48q8dhtsUpNl6OD627hR; Domain=dropbox.com;
> expires=Wed, 22 Jun 2022 13:25:44 GMT; httponly; Path=/; secure
> Set-Cookie: __Host-js_csrf=dFl-48q8dhtsUpNl6OD627hR; expires=Wed, 22
> Jun 2022 13:25:44 GMT; Path=/; secure
> X-Content-Type-Options: nosniff
> X-Dropbox-Request-Id: 95cdc3818032b5355abb335726e7093e
> X-Frame-Options: DENY
> X-Robots-Tag: noindex, nofollow, noimageindex
> X-Xss-Protection: 1; mode=block
> Strict-Transport-Security: max-age=15552000; includeSubDomains
>
> HTTP/1.1 302 Found
> Server: nginx
> Date: Sun, 23 Jun 2019 13:25:44 GMT
> Content-Type: text/html; charset=utf-8
> Connection: keep-alive
> Cache-Control: no-cache
> Content-Security-Policy: sandbox
> Location:
> https://<someloc>.dl.dropboxusercontent.com/cd/0/inline/AjUbqHAmT0GH4EvTrTsHe6LtsGUcmKcxq__N0fAkvSybpqlIr_zo7f49YDzIcmr7O4cWwqH1Vp85Xd2Ro7S_wrWhux4LK2u_sNbF7-2qVwGLaw/file#
> Pragma: no-cache
> Referrer-Policy: origin-when-cross-origin
> Set-Cookie: locale=en; Domain=dropbox.com; expires=Fri, 21 Jun 2024
> 13:25:44 GMT; Path=/; secure
> Set-Cookie: gvc=Mjg5MTM5MTgzNzcwMDQ3MTE3ODE3ODEyMTIyODUwODg4MjI5Nzkw;
> expires=Fri, 21 Jun 2024 13:25:44 GMT; httponly; Path=/; secure
> Set-Cookie: flash=; Domain=dropbox.com; expires=Sun, 23 Jun 2019
> 13:25:44 GMT; Path=/; secure
> Set-Cookie: puc=; expires=Sun, 23 Jun 2019 13:25:44 GMT; httponly;
> Path=/; secure
> Set-Cookie: bang=; Domain=dropbox.com; expires=Sun, 23 Jun 2019
> 13:25:44 GMT; Path=/; secure
> Set-Cookie: t=GVbLM33tvYrFdQl3keuX6maj; Domain=dropbox.com;
> expires=Wed, 22 Jun 2022 13:25:44 GMT; httponly; Path=/; secure
> Set-Cookie: __Host-js_csrf=GVbLM33tvYrFdQl3keuX6maj; expires=Wed, 22
> Jun 2022 13:25:44 GMT; Path=/; secure
> X-Content-Type-Options: nosniff
> X-Dropbox-Request-Id: 8196ea9b627918cdc3057982fa5254fb
> X-Frame-Options: DENY
> X-Xss-Protection: 1; mode=block
> Strict-Transport-Security: max-age=15552000; includeSubDomains
>
> HTTP/1.1 200 OK
> Server: nginx
> Date: Sun, 23 Jun 2019 13:25:44 GMT
> Content-Type: video/mp4 <<<<
> Content-Length: 92894175 <<<<<
> Connection: keep-alive
> content-disposition: inline; filename="test_out4.mp4"; filename*=UTF-
> 8''test_out4.mp4
> x-robots-tag: noindex, nofollow, noimageindex
> x-content-type-options: nosniff
> set-cookie:
> uc_session=pQdbyZ45ckq6BSRRKmfz247ugcFfboyCV55Od94quDvPkkGmSR5LfNhS8L
> f3nINL; Domain=dropboxusercontent.com; httponly; Path=/; secure
> accept-ranges: bytes
> content-security-policy: sandbox allow-forms allow-scripts allow-top-
> navigation allow-popups ; report-uri
> https://www.dropbox.com/csp_log?policy_name=blockserver-usercontent
> content-security-policy: form-action 'none' ; report-uri
> https://www.dropbox.com/csp_log?policy_name=blockserver-noscript ;
> script-src 'none'
> etag: 2d
> x-dropbox-request-id: f260f699a530d0694879d078aeffa804
> pragma: public
> cache-control: max-age=60
> referrer-policy: no-referrer
> Vary: Origin
> X-Server-Response-Time: 301
> Strict-Transport-Security: max-age=15552000; includeSubDomains
> On Sunday, June 23, 2019, 2:50:03 AM PDT, Oleg Kalnichevski <
> olegk@apache.org> wrote:
>
> On Sat, 2019-06-22 at 17:22 +0000, Krishna Sankaran wrote:
> > No, Disabling header compression did not work. One thing to
> > notice
> > is this redirect is not to another host, but a different path in
> > the
> > same server. Does it make a difference? I have attached the log
> > below.
> >
>
> Krishna,
>
> But there is no `Content-Length` header in the server response
> message.
> to begin with.
>
> Oleg
>
>
> > I got the log using the following parameters. I don't see multiple
> > get requests going out. I only one see request/response. even
> > though
> > when i use curl, i see a 301, followed by 302 followed by 200.
> > Also
> > the context.getRedirectLocations() also returns a null.
> >
> > -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Si
> > mp
> > leLog
> > -Dorg.apache.commons.logging.simplelog.showdatetime=true
> > -Dorg.apache.commons.logging.simplelog.log.org.apache.http=DEBUG
> > -Dorg.apache.commons.logging.simplelog.log.org.apache.http.wire=DEB
> > UG
> > -Dorg.apache.commons.logging.simplelog.log.org.apache.http.impl.con
> > n=
> > DEBUG
> > -Dorg.apache.commons.logging.simplelog.log.org.apache.http.client=D
> > EB
> > UG
> >
> >
> > 2019/06/22 10:13:31:796 PDT [DEBUG] RequestAddCookies - CookieSpec
> > selected: default
> > 2019/06/22 10:13:31:818 PDT [DEBUG] RequestAuthCache - Auth cache
> > not
> > set in the context
> > 2019/06/22 10:13:31:820 PDT [DEBUG]
> > PoolingHttpClientConnectionManager - Connection request: [route:
> > {s}-
> > > https://www.dropbox.com:443][total kept alive: 0; route
> > > allocated: 0
> >
> > of 2; total allocated: 0 of 20]
> > 2019/06/22 10:13:31:838 PDT [DEBUG]
> > PoolingHttpClientConnectionManager - Connection leased: [id:
> > 0][route: {s}->https://www.dropbox.com:443][total kept alive: 0;
> > route allocated: 1 of 2; total allocated: 1 of 20]
> > 2019/06/22 10:13:31:840 PDT [DEBUG] MainClientExec - Opening
> > connection {s}->https://www.dropbox.com:443
> > 2019/06/22 10:13:31:866 PDT [DEBUG]
> > DefaultHttpClientConnectionOperator - Connecting to
> > www.dropbox.com/162.125.7.1:443
> > 2019/06/22 10:13:31:866 PDT [DEBUG] SSLConnectionSocketFactory -
> > Connecting socket to www.dropbox.com/162.125.7.1:443 with timeout 0
> > 2019/06/22 10:13:31:946 PDT [DEBUG] SSLConnectionSocketFactory -
> > Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
> > 2019/06/22 10:13:31:947 PDT [DEBUG] SSLConnectionSocketFactory -
> > Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,
> > TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
> > TLS_RSA_WITH_AES_256_CBC_SHA256,
> > TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384,
> > TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384,
> > TLS_DHE_RSA_WITH_AES_256_CBC_SHA256,
> > TLS_DHE_DSS_WITH_AES_256_CBC_SHA256,
> > TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
> > TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA,
> > TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,
> > TLS_ECDH_RSA_WITH_AES_256_CBC_SHA,
> > TLS_DHE_RSA_WITH_AES_256_CBC_SHA,
> > TLS_DHE_DSS_WITH_AES_256_CBC_SHA,
> > TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,
> > TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
> > TLS_RSA_WITH_AES_128_CBC_SHA256,
> > TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256,
> > TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256,
> > TLS_DHE_RSA_WITH_AES_128_CBC_SHA256,
> > TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,
> > TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
> > TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA,
> > TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,
> > TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,
> > TLS_DHE_RSA_WITH_AES_128_CBC_SHA,
> > TLS_DHE_DSS_WITH_AES_128_CBC_SHA,
> > TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
> > TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
> > TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
> > TLS_RSA_WITH_AES_256_GCM_SHA384,
> > TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384,
> > TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384,
> > TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,
> > TLS_DHE_DSS_WITH_AES_256_GCM_SHA384,
> > TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
> > TLS_RSA_WITH_AES_128_GCM_SHA256,
> > TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256,
> > TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256,
> > TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,
> > TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
> > 2019/06/22 10:13:31:947 PDT [DEBUG] SSLConnectionSocketFactory -
> > Starting handshake
> > 2019/06/22 10:13:32:044 PDT [DEBUG] SSLConnectionSocketFactory -
> > Secure session established
> > 2019/06/22 10:13:32:045 PDT [DEBUG] SSLConnectionSocketFactory -
> > negotiated protocol: TLSv1.2
> > 2019/06/22 10:13:32:045 PDT [DEBUG] SSLConnectionSocketFactory -
> > negotiated cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
> > 2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory -
> > peer principal: CN=www.dropbox.com, O="Dropbox, Inc", L=San
> > Francisco, ST=California, C=US, SERIALNUMBER=4348296,
> > OID.1.3.6.1.4.1.311.60.2.1.2=Delaware,
> > OID.1.3.6.1.4.1.311.60.2.1.3=US, OID.2.5.4.15=Private Organization
> > 2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory -
> > peer alternative names: [www.dropbox.com, dropbox.com,
> > support.dropbox.com, live.dropbox.com, opensource.dropbox.com,
> > linux.dropbox.com, texter.dropbox.com]
> > 2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory -
> > issuer principal: CN=DigiCert SHA2 Extended Validation Server CA,
> > OU=
> > www.digicert.com, O=DigiCert Inc, C=US
> > 2019/06/22 10:13:32:050 PDT [DEBUG]
> > DefaultHttpClientConnectionOperator - Connection established
> > 192.168.1.7:50471<->162.125.7.1:443
> > 2019/06/22 10:13:32:050 PDT [DEBUG] MainClientExec - Executing
> > request HEAD /s/<somedir>/test_out4.mp4 HTTP/1.1
> > 2019/06/22 10:13:32:050 PDT [DEBUG] MainClientExec - Target auth
> > state: UNCHALLENGED
> > 2019/06/22 10:13:32:051 PDT [DEBUG] MainClientExec - Proxy auth
> > state: UNCHALLENGED
> > 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >>
> > HEAD
> > /s/<Somedire>/test_out4.mp4 HTTP/1.1
> > 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >>
> > User-Agent: MySuperUserAgent
> > 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >>
> > Host: www.dropbox.com
> > 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >>
> > Connection: Keep-Alive
> > 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >> "HEAD
> > /s/<somedir>/test_out4.mp4 HTTP/1.1[\r][\n]"
> > 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >>
> > "User-
> > Agent: MySuperUserAgent[\r][\n]"
> > 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >>
> > "Host:
> > www.dropbox.com[\r][\n]"
> > 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >>
> > "Connection: Keep-Alive[\r][\n]"
> > 2019/06/22 10:13:32:054 PDT [DEBUG] wire - http-outgoing-0 >>
> > "[\r][\n]"
> > 2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 <<
> > "HTTP/1.1 200 OK[\r][\n]"
> > 2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Server: nginx[\r][\n]"
> > 2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Date:
> > Sat, 22 Jun 2019 17:13:32 GMT[\r][\n]"
> > 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Content-Type: text/html; charset=utf-8[\r][\n]"
> > 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Connection: keep-alive[\r][\n]"
> > 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Vary:
> > Accept-Encoding[\r][\n]"
> > 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Vary:
> > Accept-Encoding[\r][\n]"
> > 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Cache-
> > Control: no-cache[\r][\n]"
> > 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Content-Security-Policy: script-src 'unsafe-eval'
> > https://www.dropbox.com/static/compiled/js/
> > https://www.dropbox.com/static/api/
> > https://www.dropbox.com/page_success/
> > https://cfl.dropboxstatic.com/static/compiled/js/
> > https://www.dropboxstatic.com/static/compiled/js/
> > https://cfl.dropboxstatic.com/static/js/
> > https://www.dropboxstatic.com/static/js/
> > https://cfl.dropboxstatic.com/static/src/dws-ensemble-appshell/
> > https://www.dropboxstatic.com/static/src/dws-ensemble-appshell/
> > https://cfl.dropboxstatic.com/static/previews/
> > https://www.dropboxstatic.com/static/previews/
> > https://cfl.dropboxstatic.com/static/api/
> > https://www.dropboxstatic.com/static/api/
> > https://cfl.dropboxstatic.com/static/cms/
> > https://www.dropboxstatic.com/static/cms/
> > https://www.google.com/recaptcha/
> > https://www.gstatic.com/recaptcha/
> > 'unsafe-inline' ; img-src https://* data: blob: ; frame-ancestors
> > 'self' ; default-src https://www.dropbox.com/playlist/
> > https://www.dropbox.com/v/s/playlist/
> > https://*.dropboxusercontent.com/p/hls_master_playlist/
> > https://*.dropboxusercontent.com/p/hls_playlist/ ; frame-src
> > https://* carousel://* dbapi-6://* dbapi-7://* dbapi-8://* itms-
> > apps://* itms-appss://* ; worker-src
> > https://www.dropbox.com/static/serviceworker/ blob: ; style-src
> > https://* 'unsafe-inline' 'unsafe-eval' ; connect-src
> > https://* ws://127.0.0.1:*/ws ; object-src 'self'
> > https://cfl.dropboxstatic.com/static/
> > https://www.dropboxstatic.com/static/
> > https://flash.dropboxstatic.com
> > https://swf.dropboxstatic.com https://dbxlocal.dropboxstatic.com
> > ;
> > media-src https://* blob: ; font-src https://* data: ; child-src
> > https://www.dropbox.com/static/serviceworker/ blob: ; form-action
> > 'self' https://www.dropbox.com/ https://dl-web.dropbox.com/
> > https://photos.dropbox.com/ https://paper.dropbox.com/
> > https://showcase.dropbox.com/ https://accounts.google.com/
> > https://api.login.yahoo.com/ https://www.hellofax.com/
> > https://app.hellofax.com/ https://www.hellosign.com/
> > https://app.hellosign.com/ https://login.yahoo.com/ ; base-uri
> > 'self'
> > ; report-uri
> >
https://www.dropbox.com/csp_log?policy_name=metaserver-whitelist[\r][\n
> > ]"
> > 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Dropbox-Streaming: V=1[\r][\n]"
> > 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Pragma: no-cache[\r][\n]"
> > 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Referrer-Policy: origin-when-cross-origin[\r][\n]"
> > 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> > Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D; expires=Mon, 22 Jul 2019
> > 17:13:32 GMT; Path=/; secure[\r][\n]"
> > 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> > Cookie: locale=en; Domain=dropbox.com; expires=Thu, 20 Jun 2024
> > 17:13:32 GMT; Path=/; secure[\r][\n]"
> > 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> > Cookie: gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D;
> > expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/;
> > secure[\r][\n]"
> > 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> > Cookie: flash=; Domain=dropbox.com; expires=Sat, 22 Jun 2019
> > 17:13:32
> > GMT; Path=/; secure[\r][\n]"
> > 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> > Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32 GMT; httponly;
> > Path=/; secure[\r][\n]"
> > 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> > Cookie: bang=; Domain=dropbox.com; expires=Sat, 22 Jun 2019
> > 17:13:32
> > GMT; Path=/; secure[\r][\n]"
> > 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> > Cookie: t=NoGNAN4y5h84gjei3tzmqbGl; Domain=dropbox.com;
> > expires=Tue,
> > 21 Jun 2022 17:13:32 GMT; httponly; Path=/; secure[\r][\n]"
> > 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> > Cookie: __Host-js_csrf=NoGNAN4y5h84gjei3tzmqbGl; expires=Tue, 21
> > Jun
> > 2022 17:13:32 GMT; Path=/; secure[\r][\n]"
> > 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> > Content-Type-Options: nosniff[\r][\n]"
> > 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> > Dropbox-Request-Id: 2d37a02490831955c27085616282ef96[\r][\n]"
> > 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> > Frame-Options: DENY[\r][\n]"
> > 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> > Robots-Tag: noindex, nofollow, noimageindex[\r][\n]"
> > 2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 << "X-
> > Xss-
> > Protection: 1; mode=block[\r][\n]"
> > 2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 <<
> > "Strict-Transport-Security: max-age=15552000;
> > includeSubDomains[\r][\n]"
> > 2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 <<
> > "[\r][\n]"
> > 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> > HTTP/1.1 200 OK
> > 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> > Server: nginx
> > 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> > Date: Sat, 22 Jun 2019 17:13:32 GMT
> > 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> > Content-Type: text/html; charset=utf-8
> > 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> > Connection: keep-alive
> > 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> > Vary: Accept-Encoding
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Vary: Accept-Encoding
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Cache-Control: no-cache
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Content-Security-Policy: script-src 'unsafe-eval'
> > https://www.dropbox.com/static/compiled/js/
> > https://www.dropbox.com/static/api/
> > https://www.dropbox.com/page_success/
> > https://cfl.dropboxstatic.com/static/compiled/js/
> > https://www.dropboxstatic.com/static/compiled/js/
> > https://cfl.dropboxstatic.com/static/js/
> > https://www.dropboxstatic.com/static/js/
> > https://cfl.dropboxstatic.com/static/src/dws-ensemble-appshell/
> > https://www.dropboxstatic.com/static/src/dws-ensemble-appshell/
> > https://cfl.dropboxstatic.com/static/previews/
> > https://www.dropboxstatic.com/static/previews/
> > https://cfl.dropboxstatic.com/static/api/
> > https://www.dropboxstatic.com/static/api/
> > https://cfl.dropboxstatic.com/static/cms/
> > https://www.dropboxstatic.com/static/cms/
> > https://www.google.com/recaptcha/
> > https://www.gstatic.com/recaptcha/
> > 'unsafe-inline' ; img-src https://* data: blob: ; frame-ancestors
> > 'self' ; default-src https://www.dropbox.com/playlist/
> > https://www.dropbox.com/v/s/playlist/
> > https://*.dropboxusercontent.com/p/hls_master_playlist/
> > https://*.dropboxusercontent.com/p/hls_playlist/ ; frame-src
> > https://* carousel://* dbapi-6://* dbapi-7://* dbapi-8://* itms-
> > apps://* itms-appss://* ; worker-src
> > https://www.dropbox.com/static/serviceworker/ blob: ; style-src
> > https://* 'unsafe-inline' 'unsafe-eval' ; connect-src
> > https://* ws://127.0.0.1:*/ws ; object-src 'self'
> > https://cfl.dropboxstatic.com/static/
> > https://www.dropboxstatic.com/static/
> > https://flash.dropboxstatic.com
> > https://swf.dropboxstatic.com https://dbxlocal.dropboxstatic.com
> > ;
> > media-src https://* blob: ; font-src https://* data: ; child-src
> > https://www.dropbox.com/static/serviceworker/ blob: ; form-action
> > 'self' https://www.dropbox.com/ https://dl-web.dropbox.com/
> > https://photos.dropbox.com/ https://paper.dropbox.com/
> > https://showcase.dropbox.com/ https://accounts.google.com/
> > https://api.login.yahoo.com/ https://www.hellofax.com/
> > https://app.hellofax.com/ https://www.hellosign.com/
> > https://app.hellosign.com/ https://login.yahoo.com/ ; base-uri
> > 'self'
> > ; report-uri
> > https://www.dropbox.com/csp_log?policy_name=metaserver-whitelist
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Dropbox-Streaming: V=1
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Pragma: no-cache
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Referrer-Policy: origin-when-cross-origin
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Set-
> > Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D; expires=Mon, 22 Jul 2019
> > 17:13:32 GMT; Path=/; secure
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Set-
> > Cookie: locale=en; Domain=dropbox.com; expires=Thu, 20 Jun 2024
> > 17:13:32 GMT; Path=/; secure
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Set-
> > Cookie: gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D;
> > expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/; secure
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Set-
> > Cookie: flash=; Domain=dropbox.com; expires=Sat, 22 Jun 2019
> > 17:13:32
> > GMT; Path=/; secure
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Set-
> > Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32 GMT; httponly;
> > Path=/; secure
> > 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> > Set-
> > Cookie: bang=; Domain=dropbox.com; expires=Sat, 22 Jun 2019
> > 17:13:32
> > GMT; Path=/; secure
> > 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 <<
> > Set-
> > Cookie: t=NoGNAN4y5h84gjei3tzmqbGl; Domain=dropbox.com;
> > expires=Tue,
> > 21 Jun 2022 17:13:32 GMT; httponly; Path=/; secure
> > 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 <<
> > Set-
> > Cookie: __Host-js_csrf=NoGNAN4y5h84gjei3tzmqbGl; expires=Tue, 21
> > Jun
> > 2022 17:13:32 GMT; Path=/; secure
> > 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> > Content-Type-Options: nosniff
> > 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> > Dropbox-Request-Id: 2d37a02490831955c27085616282ef96
> > 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> > Frame-Options: DENY
> > 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> > Robots-Tag: noindex, nofollow, noimageindex
> > 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> > Xss-Protection: 1; mode=block
> > 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 <<
> > Strict-Transport-Security: max-age=15552000; includeSubDomains
> > 2019/06/22 10:13:32:444 PDT [DEBUG] MainClientExec - Connection can
> > be kept alive indefinitely
> > 2019/06/22 10:13:32:445 PDT [DEBUG]
> > PoolingHttpClientConnectionManager - Connection [id: 0][route: {s}-
> > >
> > https://www.dropbox.com:443] can be kept alive indefinitely
> > 2019/06/22 10:13:32:445 PDT [DEBUG]
> > DefaultManagedHttpClientConnection - http-outgoing-0: set socket
> > timeout to 0
> > 2019/06/22 10:13:32:445 PDT [DEBUG]
> > PoolingHttpClientConnectionManager - Connection released: [id:
> > 0][route: {s}->https://www.dropbox.com:443][total kept alive: 1;
> > route allocated: 1 of 2; total allocated: 1 of 20]
> > 2019/06/22 10:13:32:455 PDT [WARN] ResponseProcessCookies - Invalid
> > cookie header: "Set-Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D;
> > expires=Mon, 22 Jul 2019 17:13:32 GMT; Path=/; secure". Invalid
> > 'expires' attribute: Mon, 22 Jul 2019 17:13:32 GMT
> > 2019/06/22 10:13:32:456 PDT [WARN] ResponseProcessCookies - Invalid
> > cookie header: "Set-Cookie: locale=en; Domain=dropbox.com;
> > expires=Thu, 20 Jun 2024 17:13:32 GMT; Path=/; secure". Invalid
> > 'expires' attribute: Thu, 20 Jun 2024 17:13:32 GMT
> > 2019/06/22 10:13:32:456 PDT [WARN] ResponseProcessCookies - Invalid
> > cookie header: "Set-Cookie:
> > gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D;
> > expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/; secure".
> > Invalid 'expires' attribute: Thu, 20 Jun 2024 17:13:32 GMT
> > 2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid
> > cookie header: "Set-Cookie: flash=; Domain=dropbox.com;
> > expires=Sat,
> > 22 Jun 2019 17:13:32 GMT; Path=/; secure". Invalid 'expires'
> > attribute: Sat, 22 Jun 2019 17:13:32 GMT
> > 2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid
> > cookie header: "Set-Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32
> > GMT; httponly; Path=/; secure". Invalid 'expires' attribute: Sat,
> > 22
> > Jun 2019 17:13:32 GMT
> > 2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid
> > cookie header: "Set-Cookie: bang=; Domain=dropbox.com; expires=Sat,
> > 22 Jun 2019 17:13:32 GMT; Path=/; secure". Invalid 'expires'
> > attribute: Sat, 22 Jun 2019 17:13:32 GMT
> > 2019/06/22 10:13:32:458 PDT [WARN] ResponseProcessCookies - Invalid
> > cookie header: "Set-Cookie: t=NoGNAN4y5h84gjei3tzmqbGl;
> > Domain=dropbox.com; expires=Tue, 21 Jun 2022 17:13:32 GMT;
> > httponly;
> > Path=/; secure". Invalid 'expires' attribute: Tue, 21 Jun 2022
> > 17:13:32 GMT
> > 2019/06/22 10:13:32:458 PDT [WARN] ResponseProcessCookies - Invalid
> > cookie header: "Set-Cookie: __Host-
> > js_csrf=NoGNAN4y5h84gjei3tzmqbGl;
> > expires=Tue, 21 Jun 2022 17:13:32 GMT; Path=/; secure". Invalid
> > 'expires' attribute: Tue, 21 Jun 2022 17:13:32 GMT
> >
> >
> >
> >
> > On Friday, June 21, 2019, 9:03:41 AM PDT, Oleg Kalnichevski <
> > olegk@apache.org> wrote:
> >
> > On Fri, 2019-06-21 at 00:04 +0000, Krishna Sankaran wrote:
> > > I am using the following java snippet for a HEAD request. The
> > > request
> > > gets redirected couple of times, a 301 followed by 302 and
> > > dinally
> > > gets a 200.When i use the following code snippet, i get the
> > > status
> > > as
> > > 200 OK, however all the header fields are from the 1st response
> > > (301). The Content-Length header is present in the last response,
> > > but
> > > i don't see that in the header's received by the code. Is there a
> > > knob to get the last header or all headers?
> > >
> > > private static final String SAMPLE_URL = "
> > > https://www.dropbox.com/s/<something>/test_out4.mp4";
> > >
> > > public static void main(String[] args) throws IOException,
> > > URISyntaxException {
> > > HttpClient client = HttpClientBuilder.create().build();
> > > HttpHead request = new HttpHead(new URI(SAMPLE_URL));
> > > HttpResponse response = client.execute(request);
> > > System.out.println(response.getStatusLine());
> > > for (Header header : response.getAllHeaders()) {
> > > System.out.println(header.getName() + ": " +
> > > header.getValue());
> > > }
> > > }
> > >
> > > It returns the code as 200 OK, but the Content-Type is incorrect,
> > > even
> > > the
> > > HTTP/1.1 200 OK <<< Status is 200 Server: nginx Date: Thu, 20 Jun
> > > 2019 02:22:58 GMT Content-Type: text/html; charset=utf-8 <<
> > > Content
> > > type is char <<<< Content-
> > > Length
> > > is missing!!!!!
> > >
> > > curl -I https://www.dropbox.com/s//test_out4.mp4 << Curl works
> > > correctly
> > >
> > >
> > >
> > > HTTP/1.1 301 Moved Permanently <<< Status 301 Server: nginx
> > > Date: Thu, 20 Jun 2019 02:20:50 GMT Content-Type: text/html;
> > > charset=utf-8 <<< Content type text Connection: keep-alive
> > > ....
> > > HTTP/1.1 302 Found << second redirect Server: nginx
> > > Date: Thu, 20 Jun 2019 02:36:03 GMT
> > > Content-Type: text/html; charset=utf-8
> > > ....
> > > HTTP/1.1 200 OK <<< Status finally 200 Server: nginx
> > > Date: Thu, 20 Jun 2019 02:36:04 GMT
> > > Content-Type: video/mp4 << content type correct
> > > Content-Length: 92894175 << length correct Connection: keep-
> > > alive
> > >
> > >
> >
> > Krishna
> >
> > I _suppose_ this is due automatic content compression used by
> > HttpClient per default. This can be easy to tell if you turn on
> > HttpClient context / wire logging.
> >
> > Try disabling automatic content compression.
> >
> > ---
> > CloseableHttpClient client = HttpClients.custom()
> > .disableContentCompression()
> > .build();
> > ---
> >
> > Oleg
> >
> >
> > -----------------------------------------------------------------
> > ----
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail:
> > httpclient-users-help@hc.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HTTP headers for Redirected HEAD requests
Posted by Krishna Sankaran <kr...@yahoo.com.INVALID>.
Attached the curl output for the same URL.
There is a 301, followed by 302 followed by 200.
Content-Type: text/html; charset=utf-8 is seen for the 301 and 302. and there is no Content-Length
But finally, for the 200 i see the correct Content-Type and Length.
Content-Type: video/mp4 <<<<
Content-Length: 92894175 <<<<<
I set the user agent to "some other string" to see if the server is doing something strange based on UserAgent.
curl -L -I https://www.dropbox.com/s/<sommedir>/test_out4.mp4
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Sun, 23 Jun 2019 13:25:44 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Cache-Control: no-cache
Content-Security-Policy: sandbox
Location: /s/raw/<somedir>/test_out4.mp4
Pragma: no-cache
Referrer-Policy: origin-when-cross-origin
Set-Cookie: locale=en; Domain=dropbox.com; expires=Fri, 21 Jun 2024 13:25:44 GMT; Path=/; secure
Set-Cookie: gvc=MTA3NDcwOTg3MzE2NzIyNTE1MjMxNTg0OTM2OTA4NDcyMDExNzcx; expires=Fri, 21 Jun 2024 13:25:44 GMT; httponly; Path=/; secure
Set-Cookie: flash=; Domain=dropbox.com; expires=Sun, 23 Jun 2019 13:25:44 GMT; Path=/; secure
Set-Cookie: puc=; expires=Sun, 23 Jun 2019 13:25:44 GMT; httponly; Path=/; secure
Set-Cookie: bang=; Domain=dropbox.com; expires=Sun, 23 Jun 2019 13:25:44 GMT; Path=/; secure
Set-Cookie: t=dFl-48q8dhtsUpNl6OD627hR; Domain=dropbox.com; expires=Wed, 22 Jun 2022 13:25:44 GMT; httponly; Path=/; secure
Set-Cookie: __Host-js_csrf=dFl-48q8dhtsUpNl6OD627hR; expires=Wed, 22 Jun 2022 13:25:44 GMT; Path=/; secure
X-Content-Type-Options: nosniff
X-Dropbox-Request-Id: 95cdc3818032b5355abb335726e7093e
X-Frame-Options: DENY
X-Robots-Tag: noindex, nofollow, noimageindex
X-Xss-Protection: 1; mode=block
Strict-Transport-Security: max-age=15552000; includeSubDomains
HTTP/1.1 302 Found
Server: nginx
Date: Sun, 23 Jun 2019 13:25:44 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Cache-Control: no-cache
Content-Security-Policy: sandbox
Location: https://<someloc>.dl.dropboxusercontent.com/cd/0/inline/AjUbqHAmT0GH4EvTrTsHe6LtsGUcmKcxq__N0fAkvSybpqlIr_zo7f49YDzIcmr7O4cWwqH1Vp85Xd2Ro7S_wrWhux4LK2u_sNbF7-2qVwGLaw/file#
Pragma: no-cache
Referrer-Policy: origin-when-cross-origin
Set-Cookie: locale=en; Domain=dropbox.com; expires=Fri, 21 Jun 2024 13:25:44 GMT; Path=/; secure
Set-Cookie: gvc=Mjg5MTM5MTgzNzcwMDQ3MTE3ODE3ODEyMTIyODUwODg4MjI5Nzkw; expires=Fri, 21 Jun 2024 13:25:44 GMT; httponly; Path=/; secure
Set-Cookie: flash=; Domain=dropbox.com; expires=Sun, 23 Jun 2019 13:25:44 GMT; Path=/; secure
Set-Cookie: puc=; expires=Sun, 23 Jun 2019 13:25:44 GMT; httponly; Path=/; secure
Set-Cookie: bang=; Domain=dropbox.com; expires=Sun, 23 Jun 2019 13:25:44 GMT; Path=/; secure
Set-Cookie: t=GVbLM33tvYrFdQl3keuX6maj; Domain=dropbox.com; expires=Wed, 22 Jun 2022 13:25:44 GMT; httponly; Path=/; secure
Set-Cookie: __Host-js_csrf=GVbLM33tvYrFdQl3keuX6maj; expires=Wed, 22 Jun 2022 13:25:44 GMT; Path=/; secure
X-Content-Type-Options: nosniff
X-Dropbox-Request-Id: 8196ea9b627918cdc3057982fa5254fb
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block
Strict-Transport-Security: max-age=15552000; includeSubDomains
HTTP/1.1 200 OK
Server: nginx
Date: Sun, 23 Jun 2019 13:25:44 GMT
Content-Type: video/mp4 <<<<
Content-Length: 92894175 <<<<<
Connection: keep-alive
content-disposition: inline; filename="test_out4.mp4"; filename*=UTF-8''test_out4.mp4
x-robots-tag: noindex, nofollow, noimageindex
x-content-type-options: nosniff
set-cookie: uc_session=pQdbyZ45ckq6BSRRKmfz247ugcFfboyCV55Od94quDvPkkGmSR5LfNhS8Lf3nINL; Domain=dropboxusercontent.com; httponly; Path=/; secure
accept-ranges: bytes
content-security-policy: sandbox allow-forms allow-scripts allow-top-navigation allow-popups ; report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-usercontent
content-security-policy: form-action 'none' ; report-uri https://www.dropbox.com/csp_log?policy_name=blockserver-noscript ; script-src 'none'
etag: 2d
x-dropbox-request-id: f260f699a530d0694879d078aeffa804
pragma: public
cache-control: max-age=60
referrer-policy: no-referrer
Vary: Origin
X-Server-Response-Time: 301
Strict-Transport-Security: max-age=15552000; includeSubDomains
On Sunday, June 23, 2019, 2:50:03 AM PDT, Oleg Kalnichevski <ol...@apache.org> wrote:
On Sat, 2019-06-22 at 17:22 +0000, Krishna Sankaran wrote:
> No, Disabling header compression did not work. One thing to notice
> is this redirect is not to another host, but a different path in the
> same server. Does it make a difference? I have attached the log
> below.
>
Krishna,
But there is no `Content-Length` header in the server response message.
to begin with.
Oleg
> I got the log using the following parameters. I don't see multiple
> get requests going out. I only one see request/response. even though
> when i use curl, i see a 301, followed by 302 followed by 200. Also
> the context.getRedirectLocations() also returns a null.
>
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Simp
> leLog
> -Dorg.apache.commons.logging.simplelog.showdatetime=true
> -Dorg.apache.commons.logging.simplelog.log.org.apache.http=DEBUG
> -Dorg.apache.commons.logging.simplelog.log.org.apache.http.wire=DEBUG
> -Dorg.apache.commons.logging.simplelog.log.org.apache.http.impl.conn=
> DEBUG
> -Dorg.apache.commons.logging.simplelog.log.org.apache.http.client=DEB
> UG
>
>
> 2019/06/22 10:13:31:796 PDT [DEBUG] RequestAddCookies - CookieSpec
> selected: default
> 2019/06/22 10:13:31:818 PDT [DEBUG] RequestAuthCache - Auth cache not
> set in the context
> 2019/06/22 10:13:31:820 PDT [DEBUG]
> PoolingHttpClientConnectionManager - Connection request: [route: {s}-
> >https://www.dropbox.com:443][total kept alive: 0; route allocated: 0
> of 2; total allocated: 0 of 20]
> 2019/06/22 10:13:31:838 PDT [DEBUG]
> PoolingHttpClientConnectionManager - Connection leased: [id:
> 0][route: {s}->https://www.dropbox.com:443][total kept alive: 0;
> route allocated: 1 of 2; total allocated: 1 of 20]
> 2019/06/22 10:13:31:840 PDT [DEBUG] MainClientExec - Opening
> connection {s}->https://www.dropbox.com:443
> 2019/06/22 10:13:31:866 PDT [DEBUG]
> DefaultHttpClientConnectionOperator - Connecting to
> www.dropbox.com/162.125.7.1:443
> 2019/06/22 10:13:31:866 PDT [DEBUG] SSLConnectionSocketFactory -
> Connecting socket to www.dropbox.com/162.125.7.1:443 with timeout 0
> 2019/06/22 10:13:31:946 PDT [DEBUG] SSLConnectionSocketFactory -
> Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
> 2019/06/22 10:13:31:947 PDT [DEBUG] SSLConnectionSocketFactory -
> Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,
> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
> TLS_RSA_WITH_AES_256_CBC_SHA256,
> TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384,
> TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384,
> TLS_DHE_RSA_WITH_AES_256_CBC_SHA256,
> TLS_DHE_DSS_WITH_AES_256_CBC_SHA256,
> TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA,
> TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,
> TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA,
> TLS_DHE_DSS_WITH_AES_256_CBC_SHA,
> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,
> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
> TLS_RSA_WITH_AES_128_CBC_SHA256,
> TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256,
> TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256,
> TLS_DHE_RSA_WITH_AES_128_CBC_SHA256,
> TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,
> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA,
> TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,
> TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA,
> TLS_DHE_DSS_WITH_AES_128_CBC_SHA,
> TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
> TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
> TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
> TLS_RSA_WITH_AES_256_GCM_SHA384,
> TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384,
> TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384,
> TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,
> TLS_DHE_DSS_WITH_AES_256_GCM_SHA384,
> TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
> TLS_RSA_WITH_AES_128_GCM_SHA256,
> TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256,
> TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256,
> TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,
> TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
> 2019/06/22 10:13:31:947 PDT [DEBUG] SSLConnectionSocketFactory -
> Starting handshake
> 2019/06/22 10:13:32:044 PDT [DEBUG] SSLConnectionSocketFactory -
> Secure session established
> 2019/06/22 10:13:32:045 PDT [DEBUG] SSLConnectionSocketFactory -
> negotiated protocol: TLSv1.2
> 2019/06/22 10:13:32:045 PDT [DEBUG] SSLConnectionSocketFactory -
> negotiated cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
> 2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory -
> peer principal: CN=www.dropbox.com, O="Dropbox, Inc", L=San
> Francisco, ST=California, C=US, SERIALNUMBER=4348296,
> OID.1.3.6.1.4.1.311.60.2.1.2=Delaware,
> OID.1.3.6.1.4.1.311.60.2.1.3=US, OID.2.5.4.15=Private Organization
> 2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory -
> peer alternative names: [www.dropbox.com, dropbox.com,
> support.dropbox.com, live.dropbox.com, opensource.dropbox.com,
> linux.dropbox.com, texter.dropbox.com]
> 2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory -
> issuer principal: CN=DigiCert SHA2 Extended Validation Server CA, OU=
> www.digicert.com, O=DigiCert Inc, C=US
> 2019/06/22 10:13:32:050 PDT [DEBUG]
> DefaultHttpClientConnectionOperator - Connection established
> 192.168.1.7:50471<->162.125.7.1:443
> 2019/06/22 10:13:32:050 PDT [DEBUG] MainClientExec - Executing
> request HEAD /s/<somedir>/test_out4.mp4 HTTP/1.1
> 2019/06/22 10:13:32:050 PDT [DEBUG] MainClientExec - Target auth
> state: UNCHALLENGED
> 2019/06/22 10:13:32:051 PDT [DEBUG] MainClientExec - Proxy auth
> state: UNCHALLENGED
> 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >> HEAD
> /s/<Somedire>/test_out4.mp4 HTTP/1.1
> 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >>
> User-Agent: MySuperUserAgent
> 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >>
> Host: www.dropbox.com
> 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >>
> Connection: Keep-Alive
> 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >> "HEAD
> /s/<somedir>/test_out4.mp4 HTTP/1.1[\r][\n]"
> 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >> "User-
> Agent: MySuperUserAgent[\r][\n]"
> 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >> "Host:
> www.dropbox.com[\r][\n]"
> 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >>
> "Connection: Keep-Alive[\r][\n]"
> 2019/06/22 10:13:32:054 PDT [DEBUG] wire - http-outgoing-0 >>
> "[\r][\n]"
> 2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 <<
> "HTTP/1.1 200 OK[\r][\n]"
> 2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 <<
> "Server: nginx[\r][\n]"
> 2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 << "Date:
> Sat, 22 Jun 2019 17:13:32 GMT[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> "Content-Type: text/html; charset=utf-8[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> "Connection: keep-alive[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Vary:
> Accept-Encoding[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Vary:
> Accept-Encoding[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Cache-
> Control: no-cache[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> "Content-Security-Policy: script-src 'unsafe-eval'
> https://www.dropbox.com/static/compiled/js/
> https://www.dropbox.com/static/api/
> https://www.dropbox.com/page_success/
> https://cfl.dropboxstatic.com/static/compiled/js/
> https://www.dropboxstatic.com/static/compiled/js/
> https://cfl.dropboxstatic.com/static/js/
> https://www.dropboxstatic.com/static/js/
> https://cfl.dropboxstatic.com/static/src/dws-ensemble-appshell/
> https://www.dropboxstatic.com/static/src/dws-ensemble-appshell/
> https://cfl.dropboxstatic.com/static/previews/
> https://www.dropboxstatic.com/static/previews/
> https://cfl.dropboxstatic.com/static/api/
> https://www.dropboxstatic.com/static/api/
> https://cfl.dropboxstatic.com/static/cms/
> https://www.dropboxstatic.com/static/cms/
> https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/
> 'unsafe-inline' ; img-src https://* data: blob: ; frame-ancestors
> 'self' ; default-src https://www.dropbox.com/playlist/
> https://www.dropbox.com/v/s/playlist/
> https://*.dropboxusercontent.com/p/hls_master_playlist/
> https://*.dropboxusercontent.com/p/hls_playlist/ ; frame-src
> https://* carousel://* dbapi-6://* dbapi-7://* dbapi-8://* itms-
> apps://* itms-appss://* ; worker-src
> https://www.dropbox.com/static/serviceworker/ blob: ; style-src
> https://* 'unsafe-inline' 'unsafe-eval' ; connect-src
> https://* ws://127.0.0.1:*/ws ; object-src 'self'
> https://cfl.dropboxstatic.com/static/
> https://www.dropboxstatic.com/static/ https://flash.dropboxstatic.com
> https://swf.dropboxstatic.com https://dbxlocal.dropboxstatic.com ;
> media-src https://* blob: ; font-src https://* data: ; child-src
> https://www.dropbox.com/static/serviceworker/ blob: ; form-action
> 'self' https://www.dropbox.com/ https://dl-web.dropbox.com/
> https://photos.dropbox.com/ https://paper.dropbox.com/
> https://showcase.dropbox.com/ https://accounts.google.com/
> https://api.login.yahoo.com/ https://www.hellofax.com/
> https://app.hellofax.com/ https://www.hellosign.com/
> https://app.hellosign.com/ https://login.yahoo.com/ ; base-uri 'self'
> ; report-uri
> https://www.dropbox.com/csp_log?policy_name=metaserver-whitelist[\r][\n
> ]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 <<
> "Dropbox-Streaming: V=1[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 <<
> "Pragma: no-cache[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 <<
> "Referrer-Policy: origin-when-cross-origin[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D; expires=Mon, 22 Jul 2019
> 17:13:32 GMT; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: locale=en; Domain=dropbox.com; expires=Thu, 20 Jun 2024
> 17:13:32 GMT; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D;
> expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/;
> secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: flash=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32
> GMT; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32 GMT; httponly;
> Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: bang=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32
> GMT; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: t=NoGNAN4y5h84gjei3tzmqbGl; Domain=dropbox.com; expires=Tue,
> 21 Jun 2022 17:13:32 GMT; httponly; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: __Host-js_csrf=NoGNAN4y5h84gjei3tzmqbGl; expires=Tue, 21 Jun
> 2022 17:13:32 GMT; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> Content-Type-Options: nosniff[\r][\n]"
> 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> Dropbox-Request-Id: 2d37a02490831955c27085616282ef96[\r][\n]"
> 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> Frame-Options: DENY[\r][\n]"
> 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> Robots-Tag: noindex, nofollow, noimageindex[\r][\n]"
> 2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 << "X-Xss-
> Protection: 1; mode=block[\r][\n]"
> 2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 <<
> "Strict-Transport-Security: max-age=15552000;
> includeSubDomains[\r][\n]"
> 2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 <<
> "[\r][\n]"
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> HTTP/1.1 200 OK
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> Server: nginx
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> Date: Sat, 22 Jun 2019 17:13:32 GMT
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> Content-Type: text/html; charset=utf-8
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> Connection: keep-alive
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> Vary: Accept-Encoding
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Vary: Accept-Encoding
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Cache-Control: no-cache
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Content-Security-Policy: script-src 'unsafe-eval'
> https://www.dropbox.com/static/compiled/js/
> https://www.dropbox.com/static/api/
> https://www.dropbox.com/page_success/
> https://cfl.dropboxstatic.com/static/compiled/js/
> https://www.dropboxstatic.com/static/compiled/js/
> https://cfl.dropboxstatic.com/static/js/
> https://www.dropboxstatic.com/static/js/
> https://cfl.dropboxstatic.com/static/src/dws-ensemble-appshell/
> https://www.dropboxstatic.com/static/src/dws-ensemble-appshell/
> https://cfl.dropboxstatic.com/static/previews/
> https://www.dropboxstatic.com/static/previews/
> https://cfl.dropboxstatic.com/static/api/
> https://www.dropboxstatic.com/static/api/
> https://cfl.dropboxstatic.com/static/cms/
> https://www.dropboxstatic.com/static/cms/
> https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/
> 'unsafe-inline' ; img-src https://* data: blob: ; frame-ancestors
> 'self' ; default-src https://www.dropbox.com/playlist/
> https://www.dropbox.com/v/s/playlist/
> https://*.dropboxusercontent.com/p/hls_master_playlist/
> https://*.dropboxusercontent.com/p/hls_playlist/ ; frame-src
> https://* carousel://* dbapi-6://* dbapi-7://* dbapi-8://* itms-
> apps://* itms-appss://* ; worker-src
> https://www.dropbox.com/static/serviceworker/ blob: ; style-src
> https://* 'unsafe-inline' 'unsafe-eval' ; connect-src
> https://* ws://127.0.0.1:*/ws ; object-src 'self'
> https://cfl.dropboxstatic.com/static/
> https://www.dropboxstatic.com/static/ https://flash.dropboxstatic.com
> https://swf.dropboxstatic.com https://dbxlocal.dropboxstatic.com ;
> media-src https://* blob: ; font-src https://* data: ; child-src
> https://www.dropbox.com/static/serviceworker/ blob: ; form-action
> 'self' https://www.dropbox.com/ https://dl-web.dropbox.com/
> https://photos.dropbox.com/ https://paper.dropbox.com/
> https://showcase.dropbox.com/ https://accounts.google.com/
> https://api.login.yahoo.com/ https://www.hellofax.com/
> https://app.hellofax.com/ https://www.hellosign.com/
> https://app.hellosign.com/ https://login.yahoo.com/ ; base-uri 'self'
> ; report-uri
> https://www.dropbox.com/csp_log?policy_name=metaserver-whitelist
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Dropbox-Streaming: V=1
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Pragma: no-cache
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Referrer-Policy: origin-when-cross-origin
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D; expires=Mon, 22 Jul 2019
> 17:13:32 GMT; Path=/; secure
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: locale=en; Domain=dropbox.com; expires=Thu, 20 Jun 2024
> 17:13:32 GMT; Path=/; secure
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D;
> expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/; secure
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: flash=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32
> GMT; Path=/; secure
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32 GMT; httponly;
> Path=/; secure
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: bang=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32
> GMT; Path=/; secure
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: t=NoGNAN4y5h84gjei3tzmqbGl; Domain=dropbox.com; expires=Tue,
> 21 Jun 2022 17:13:32 GMT; httponly; Path=/; secure
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: __Host-js_csrf=NoGNAN4y5h84gjei3tzmqbGl; expires=Tue, 21 Jun
> 2022 17:13:32 GMT; Path=/; secure
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> Content-Type-Options: nosniff
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> Dropbox-Request-Id: 2d37a02490831955c27085616282ef96
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> Frame-Options: DENY
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> Robots-Tag: noindex, nofollow, noimageindex
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> Xss-Protection: 1; mode=block
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 <<
> Strict-Transport-Security: max-age=15552000; includeSubDomains
> 2019/06/22 10:13:32:444 PDT [DEBUG] MainClientExec - Connection can
> be kept alive indefinitely
> 2019/06/22 10:13:32:445 PDT [DEBUG]
> PoolingHttpClientConnectionManager - Connection [id: 0][route: {s}->
> https://www.dropbox.com:443] can be kept alive indefinitely
> 2019/06/22 10:13:32:445 PDT [DEBUG]
> DefaultManagedHttpClientConnection - http-outgoing-0: set socket
> timeout to 0
> 2019/06/22 10:13:32:445 PDT [DEBUG]
> PoolingHttpClientConnectionManager - Connection released: [id:
> 0][route: {s}->https://www.dropbox.com:443][total kept alive: 1;
> route allocated: 1 of 2; total allocated: 1 of 20]
> 2019/06/22 10:13:32:455 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D;
> expires=Mon, 22 Jul 2019 17:13:32 GMT; Path=/; secure". Invalid
> 'expires' attribute: Mon, 22 Jul 2019 17:13:32 GMT
> 2019/06/22 10:13:32:456 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: locale=en; Domain=dropbox.com;
> expires=Thu, 20 Jun 2024 17:13:32 GMT; Path=/; secure". Invalid
> 'expires' attribute: Thu, 20 Jun 2024 17:13:32 GMT
> 2019/06/22 10:13:32:456 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie:
> gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D;
> expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/; secure".
> Invalid 'expires' attribute: Thu, 20 Jun 2024 17:13:32 GMT
> 2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: flash=; Domain=dropbox.com; expires=Sat,
> 22 Jun 2019 17:13:32 GMT; Path=/; secure". Invalid 'expires'
> attribute: Sat, 22 Jun 2019 17:13:32 GMT
> 2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32
> GMT; httponly; Path=/; secure". Invalid 'expires' attribute: Sat, 22
> Jun 2019 17:13:32 GMT
> 2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: bang=; Domain=dropbox.com; expires=Sat,
> 22 Jun 2019 17:13:32 GMT; Path=/; secure". Invalid 'expires'
> attribute: Sat, 22 Jun 2019 17:13:32 GMT
> 2019/06/22 10:13:32:458 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: t=NoGNAN4y5h84gjei3tzmqbGl;
> Domain=dropbox.com; expires=Tue, 21 Jun 2022 17:13:32 GMT; httponly;
> Path=/; secure". Invalid 'expires' attribute: Tue, 21 Jun 2022
> 17:13:32 GMT
> 2019/06/22 10:13:32:458 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: __Host-js_csrf=NoGNAN4y5h84gjei3tzmqbGl;
> expires=Tue, 21 Jun 2022 17:13:32 GMT; Path=/; secure". Invalid
> 'expires' attribute: Tue, 21 Jun 2022 17:13:32 GMT
>
>
>
>
> On Friday, June 21, 2019, 9:03:41 AM PDT, Oleg Kalnichevski <
> olegk@apache.org> wrote:
>
> On Fri, 2019-06-21 at 00:04 +0000, Krishna Sankaran wrote:
> > I am using the following java snippet for a HEAD request. The
> > request
> > gets redirected couple of times, a 301 followed by 302 and dinally
> > gets a 200.When i use the following code snippet, i get the status
> > as
> > 200 OK, however all the header fields are from the 1st response
> > (301). The Content-Length header is present in the last response,
> > but
> > i don't see that in the header's received by the code. Is there a
> > knob to get the last header or all headers?
> >
> > private static final String SAMPLE_URL = "
> > https://www.dropbox.com/s/<something>/test_out4.mp4";
> >
> > public static void main(String[] args) throws IOException,
> > URISyntaxException {
> > HttpClient client = HttpClientBuilder.create().build();
> > HttpHead request = new HttpHead(new URI(SAMPLE_URL));
> > HttpResponse response = client.execute(request);
> > System.out.println(response.getStatusLine());
> > for (Header header : response.getAllHeaders()) {
> > System.out.println(header.getName() + ": " +
> > header.getValue());
> > }
> > }
> >
> > It returns the code as 200 OK, but the Content-Type is incorrect,
> > even
> > the
> > HTTP/1.1 200 OK <<< Status is 200 Server: nginx Date: Thu, 20 Jun
> > 2019 02:22:58 GMT Content-Type: text/html; charset=utf-8 << Content
> > type is char <<<< Content-
> > Length
> > is missing!!!!!
> >
> > curl -I https://www.dropbox.com/s//test_out4.mp4 << Curl works
> > correctly
> >
> >
> >
> > HTTP/1.1 301 Moved Permanently <<< Status 301 Server: nginx
> > Date: Thu, 20 Jun 2019 02:20:50 GMT Content-Type: text/html;
> > charset=utf-8 <<< Content type text Connection: keep-alive
> > ....
> > HTTP/1.1 302 Found << second redirect Server: nginx
> > Date: Thu, 20 Jun 2019 02:36:03 GMT
> > Content-Type: text/html; charset=utf-8
> > ....
> > HTTP/1.1 200 OK <<< Status finally 200 Server: nginx
> > Date: Thu, 20 Jun 2019 02:36:04 GMT
> > Content-Type: video/mp4 << content type correct
> > Content-Length: 92894175 << length correct Connection: keep-alive
> >
> >
>
> Krishna
>
> I _suppose_ this is due automatic content compression used by
> HttpClient per default. This can be easy to tell if you turn on
> HttpClient context / wire logging.
>
> Try disabling automatic content compression.
>
> ---
> CloseableHttpClient client = HttpClients.custom()
> .disableContentCompression()
> .build();
> ---
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HTTP headers for Redirected HEAD requests
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sat, 2019-06-22 at 17:22 +0000, Krishna Sankaran wrote:
> No, Disabling header compression did not work. One thing to notice
> is this redirect is not to another host, but a different path in the
> same server. Does it make a difference? I have attached the log
> below.
>
Krishna,
But there is no `Content-Length` header in the server response message.
to begin with.
Oleg
> I got the log using the following parameters. I don't see multiple
> get requests going out. I only one see request/response. even though
> when i use curl, i see a 301, followed by 302 followed by 200. Also
> the context.getRedirectLocations() also returns a null.
>
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Simp
> leLog
> -Dorg.apache.commons.logging.simplelog.showdatetime=true
> -Dorg.apache.commons.logging.simplelog.log.org.apache.http=DEBUG
> -Dorg.apache.commons.logging.simplelog.log.org.apache.http.wire=DEBUG
> -Dorg.apache.commons.logging.simplelog.log.org.apache.http.impl.conn=
> DEBUG
> -Dorg.apache.commons.logging.simplelog.log.org.apache.http.client=DEB
> UG
>
>
> 2019/06/22 10:13:31:796 PDT [DEBUG] RequestAddCookies - CookieSpec
> selected: default
> 2019/06/22 10:13:31:818 PDT [DEBUG] RequestAuthCache - Auth cache not
> set in the context
> 2019/06/22 10:13:31:820 PDT [DEBUG]
> PoolingHttpClientConnectionManager - Connection request: [route: {s}-
> >https://www.dropbox.com:443][total kept alive: 0; route allocated: 0
> of 2; total allocated: 0 of 20]
> 2019/06/22 10:13:31:838 PDT [DEBUG]
> PoolingHttpClientConnectionManager - Connection leased: [id:
> 0][route: {s}->https://www.dropbox.com:443][total kept alive: 0;
> route allocated: 1 of 2; total allocated: 1 of 20]
> 2019/06/22 10:13:31:840 PDT [DEBUG] MainClientExec - Opening
> connection {s}->https://www.dropbox.com:443
> 2019/06/22 10:13:31:866 PDT [DEBUG]
> DefaultHttpClientConnectionOperator - Connecting to
> www.dropbox.com/162.125.7.1:443
> 2019/06/22 10:13:31:866 PDT [DEBUG] SSLConnectionSocketFactory -
> Connecting socket to www.dropbox.com/162.125.7.1:443 with timeout 0
> 2019/06/22 10:13:31:946 PDT [DEBUG] SSLConnectionSocketFactory -
> Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
> 2019/06/22 10:13:31:947 PDT [DEBUG] SSLConnectionSocketFactory -
> Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,
> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
> TLS_RSA_WITH_AES_256_CBC_SHA256,
> TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384,
> TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384,
> TLS_DHE_RSA_WITH_AES_256_CBC_SHA256,
> TLS_DHE_DSS_WITH_AES_256_CBC_SHA256,
> TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA,
> TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,
> TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA,
> TLS_DHE_DSS_WITH_AES_256_CBC_SHA,
> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,
> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
> TLS_RSA_WITH_AES_128_CBC_SHA256,
> TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256,
> TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256,
> TLS_DHE_RSA_WITH_AES_128_CBC_SHA256,
> TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,
> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA,
> TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,
> TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA,
> TLS_DHE_DSS_WITH_AES_128_CBC_SHA,
> TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
> TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
> TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
> TLS_RSA_WITH_AES_256_GCM_SHA384,
> TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384,
> TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384,
> TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,
> TLS_DHE_DSS_WITH_AES_256_GCM_SHA384,
> TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
> TLS_RSA_WITH_AES_128_GCM_SHA256,
> TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256,
> TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256,
> TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,
> TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
> 2019/06/22 10:13:31:947 PDT [DEBUG] SSLConnectionSocketFactory -
> Starting handshake
> 2019/06/22 10:13:32:044 PDT [DEBUG] SSLConnectionSocketFactory -
> Secure session established
> 2019/06/22 10:13:32:045 PDT [DEBUG] SSLConnectionSocketFactory -
> negotiated protocol: TLSv1.2
> 2019/06/22 10:13:32:045 PDT [DEBUG] SSLConnectionSocketFactory -
> negotiated cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
> 2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory -
> peer principal: CN=www.dropbox.com, O="Dropbox, Inc", L=San
> Francisco, ST=California, C=US, SERIALNUMBER=4348296,
> OID.1.3.6.1.4.1.311.60.2.1.2=Delaware,
> OID.1.3.6.1.4.1.311.60.2.1.3=US, OID.2.5.4.15=Private Organization
> 2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory -
> peer alternative names: [www.dropbox.com, dropbox.com,
> support.dropbox.com, live.dropbox.com, opensource.dropbox.com,
> linux.dropbox.com, texter.dropbox.com]
> 2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory -
> issuer principal: CN=DigiCert SHA2 Extended Validation Server CA, OU=
> www.digicert.com, O=DigiCert Inc, C=US
> 2019/06/22 10:13:32:050 PDT [DEBUG]
> DefaultHttpClientConnectionOperator - Connection established
> 192.168.1.7:50471<->162.125.7.1:443
> 2019/06/22 10:13:32:050 PDT [DEBUG] MainClientExec - Executing
> request HEAD /s/<somedir>/test_out4.mp4 HTTP/1.1
> 2019/06/22 10:13:32:050 PDT [DEBUG] MainClientExec - Target auth
> state: UNCHALLENGED
> 2019/06/22 10:13:32:051 PDT [DEBUG] MainClientExec - Proxy auth
> state: UNCHALLENGED
> 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >> HEAD
> /s/<Somedire>/test_out4.mp4 HTTP/1.1
> 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >>
> User-Agent: MySuperUserAgent
> 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >>
> Host: www.dropbox.com
> 2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >>
> Connection: Keep-Alive
> 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >> "HEAD
> /s/<somedir>/test_out4.mp4 HTTP/1.1[\r][\n]"
> 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >> "User-
> Agent: MySuperUserAgent[\r][\n]"
> 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >> "Host:
> www.dropbox.com[\r][\n]"
> 2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >>
> "Connection: Keep-Alive[\r][\n]"
> 2019/06/22 10:13:32:054 PDT [DEBUG] wire - http-outgoing-0 >>
> "[\r][\n]"
> 2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 <<
> "HTTP/1.1 200 OK[\r][\n]"
> 2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 <<
> "Server: nginx[\r][\n]"
> 2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 << "Date:
> Sat, 22 Jun 2019 17:13:32 GMT[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> "Content-Type: text/html; charset=utf-8[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> "Connection: keep-alive[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Vary:
> Accept-Encoding[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Vary:
> Accept-Encoding[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Cache-
> Control: no-cache[\r][\n]"
> 2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 <<
> "Content-Security-Policy: script-src 'unsafe-eval'
> https://www.dropbox.com/static/compiled/js/
> https://www.dropbox.com/static/api/
> https://www.dropbox.com/page_success/
> https://cfl.dropboxstatic.com/static/compiled/js/
> https://www.dropboxstatic.com/static/compiled/js/
> https://cfl.dropboxstatic.com/static/js/
> https://www.dropboxstatic.com/static/js/
> https://cfl.dropboxstatic.com/static/src/dws-ensemble-appshell/
> https://www.dropboxstatic.com/static/src/dws-ensemble-appshell/
> https://cfl.dropboxstatic.com/static/previews/
> https://www.dropboxstatic.com/static/previews/
> https://cfl.dropboxstatic.com/static/api/
> https://www.dropboxstatic.com/static/api/
> https://cfl.dropboxstatic.com/static/cms/
> https://www.dropboxstatic.com/static/cms/
> https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/
> 'unsafe-inline' ; img-src https://* data: blob: ; frame-ancestors
> 'self' ; default-src https://www.dropbox.com/playlist/
> https://www.dropbox.com/v/s/playlist/
> https://*.dropboxusercontent.com/p/hls_master_playlist/
> https://*.dropboxusercontent.com/p/hls_playlist/ ; frame-src
> https://* carousel://* dbapi-6://* dbapi-7://* dbapi-8://* itms-
> apps://* itms-appss://* ; worker-src
> https://www.dropbox.com/static/serviceworker/ blob: ; style-src
> https://* 'unsafe-inline' 'unsafe-eval' ; connect-src
> https://* ws://127.0.0.1:*/ws ; object-src 'self'
> https://cfl.dropboxstatic.com/static/
> https://www.dropboxstatic.com/static/ https://flash.dropboxstatic.com
> https://swf.dropboxstatic.com https://dbxlocal.dropboxstatic.com ;
> media-src https://* blob: ; font-src https://* data: ; child-src
> https://www.dropbox.com/static/serviceworker/ blob: ; form-action
> 'self' https://www.dropbox.com/ https://dl-web.dropbox.com/
> https://photos.dropbox.com/ https://paper.dropbox.com/
> https://showcase.dropbox.com/ https://accounts.google.com/
> https://api.login.yahoo.com/ https://www.hellofax.com/
> https://app.hellofax.com/ https://www.hellosign.com/
> https://app.hellosign.com/ https://login.yahoo.com/ ; base-uri 'self'
> ; report-uri
> https://www.dropbox.com/csp_log?policy_name=metaserver-whitelist[\r][\n
> ]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 <<
> "Dropbox-Streaming: V=1[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 <<
> "Pragma: no-cache[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 <<
> "Referrer-Policy: origin-when-cross-origin[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D; expires=Mon, 22 Jul 2019
> 17:13:32 GMT; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: locale=en; Domain=dropbox.com; expires=Thu, 20 Jun 2024
> 17:13:32 GMT; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D;
> expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/;
> secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: flash=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32
> GMT; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32 GMT; httponly;
> Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: bang=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32
> GMT; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: t=NoGNAN4y5h84gjei3tzmqbGl; Domain=dropbox.com; expires=Tue,
> 21 Jun 2022 17:13:32 GMT; httponly; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-
> Cookie: __Host-js_csrf=NoGNAN4y5h84gjei3tzmqbGl; expires=Tue, 21 Jun
> 2022 17:13:32 GMT; Path=/; secure[\r][\n]"
> 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> Content-Type-Options: nosniff[\r][\n]"
> 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> Dropbox-Request-Id: 2d37a02490831955c27085616282ef96[\r][\n]"
> 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> Frame-Options: DENY[\r][\n]"
> 2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-
> Robots-Tag: noindex, nofollow, noimageindex[\r][\n]"
> 2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 << "X-Xss-
> Protection: 1; mode=block[\r][\n]"
> 2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 <<
> "Strict-Transport-Security: max-age=15552000;
> includeSubDomains[\r][\n]"
> 2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 <<
> "[\r][\n]"
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> HTTP/1.1 200 OK
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> Server: nginx
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> Date: Sat, 22 Jun 2019 17:13:32 GMT
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> Content-Type: text/html; charset=utf-8
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> Connection: keep-alive
> 2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 <<
> Vary: Accept-Encoding
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Vary: Accept-Encoding
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Cache-Control: no-cache
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Content-Security-Policy: script-src 'unsafe-eval'
> https://www.dropbox.com/static/compiled/js/
> https://www.dropbox.com/static/api/
> https://www.dropbox.com/page_success/
> https://cfl.dropboxstatic.com/static/compiled/js/
> https://www.dropboxstatic.com/static/compiled/js/
> https://cfl.dropboxstatic.com/static/js/
> https://www.dropboxstatic.com/static/js/
> https://cfl.dropboxstatic.com/static/src/dws-ensemble-appshell/
> https://www.dropboxstatic.com/static/src/dws-ensemble-appshell/
> https://cfl.dropboxstatic.com/static/previews/
> https://www.dropboxstatic.com/static/previews/
> https://cfl.dropboxstatic.com/static/api/
> https://www.dropboxstatic.com/static/api/
> https://cfl.dropboxstatic.com/static/cms/
> https://www.dropboxstatic.com/static/cms/
> https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/
> 'unsafe-inline' ; img-src https://* data: blob: ; frame-ancestors
> 'self' ; default-src https://www.dropbox.com/playlist/
> https://www.dropbox.com/v/s/playlist/
> https://*.dropboxusercontent.com/p/hls_master_playlist/
> https://*.dropboxusercontent.com/p/hls_playlist/ ; frame-src
> https://* carousel://* dbapi-6://* dbapi-7://* dbapi-8://* itms-
> apps://* itms-appss://* ; worker-src
> https://www.dropbox.com/static/serviceworker/ blob: ; style-src
> https://* 'unsafe-inline' 'unsafe-eval' ; connect-src
> https://* ws://127.0.0.1:*/ws ; object-src 'self'
> https://cfl.dropboxstatic.com/static/
> https://www.dropboxstatic.com/static/ https://flash.dropboxstatic.com
> https://swf.dropboxstatic.com https://dbxlocal.dropboxstatic.com ;
> media-src https://* blob: ; font-src https://* data: ; child-src
> https://www.dropbox.com/static/serviceworker/ blob: ; form-action
> 'self' https://www.dropbox.com/ https://dl-web.dropbox.com/
> https://photos.dropbox.com/ https://paper.dropbox.com/
> https://showcase.dropbox.com/ https://accounts.google.com/
> https://api.login.yahoo.com/ https://www.hellofax.com/
> https://app.hellofax.com/ https://www.hellosign.com/
> https://app.hellosign.com/ https://login.yahoo.com/ ; base-uri 'self'
> ; report-uri
> https://www.dropbox.com/csp_log?policy_name=metaserver-whitelist
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Dropbox-Streaming: V=1
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Pragma: no-cache
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 <<
> Referrer-Policy: origin-when-cross-origin
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D; expires=Mon, 22 Jul 2019
> 17:13:32 GMT; Path=/; secure
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: locale=en; Domain=dropbox.com; expires=Thu, 20 Jun 2024
> 17:13:32 GMT; Path=/; secure
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D;
> expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/; secure
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: flash=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32
> GMT; Path=/; secure
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32 GMT; httponly;
> Path=/; secure
> 2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: bang=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32
> GMT; Path=/; secure
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: t=NoGNAN4y5h84gjei3tzmqbGl; Domain=dropbox.com; expires=Tue,
> 21 Jun 2022 17:13:32 GMT; httponly; Path=/; secure
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << Set-
> Cookie: __Host-js_csrf=NoGNAN4y5h84gjei3tzmqbGl; expires=Tue, 21 Jun
> 2022 17:13:32 GMT; Path=/; secure
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> Content-Type-Options: nosniff
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> Dropbox-Request-Id: 2d37a02490831955c27085616282ef96
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> Frame-Options: DENY
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> Robots-Tag: noindex, nofollow, noimageindex
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-
> Xss-Protection: 1; mode=block
> 2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 <<
> Strict-Transport-Security: max-age=15552000; includeSubDomains
> 2019/06/22 10:13:32:444 PDT [DEBUG] MainClientExec - Connection can
> be kept alive indefinitely
> 2019/06/22 10:13:32:445 PDT [DEBUG]
> PoolingHttpClientConnectionManager - Connection [id: 0][route: {s}->
> https://www.dropbox.com:443] can be kept alive indefinitely
> 2019/06/22 10:13:32:445 PDT [DEBUG]
> DefaultManagedHttpClientConnection - http-outgoing-0: set socket
> timeout to 0
> 2019/06/22 10:13:32:445 PDT [DEBUG]
> PoolingHttpClientConnectionManager - Connection released: [id:
> 0][route: {s}->https://www.dropbox.com:443][total kept alive: 1;
> route allocated: 1 of 2; total allocated: 1 of 20]
> 2019/06/22 10:13:32:455 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D;
> expires=Mon, 22 Jul 2019 17:13:32 GMT; Path=/; secure". Invalid
> 'expires' attribute: Mon, 22 Jul 2019 17:13:32 GMT
> 2019/06/22 10:13:32:456 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: locale=en; Domain=dropbox.com;
> expires=Thu, 20 Jun 2024 17:13:32 GMT; Path=/; secure". Invalid
> 'expires' attribute: Thu, 20 Jun 2024 17:13:32 GMT
> 2019/06/22 10:13:32:456 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie:
> gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D;
> expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/; secure".
> Invalid 'expires' attribute: Thu, 20 Jun 2024 17:13:32 GMT
> 2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: flash=; Domain=dropbox.com; expires=Sat,
> 22 Jun 2019 17:13:32 GMT; Path=/; secure". Invalid 'expires'
> attribute: Sat, 22 Jun 2019 17:13:32 GMT
> 2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32
> GMT; httponly; Path=/; secure". Invalid 'expires' attribute: Sat, 22
> Jun 2019 17:13:32 GMT
> 2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: bang=; Domain=dropbox.com; expires=Sat,
> 22 Jun 2019 17:13:32 GMT; Path=/; secure". Invalid 'expires'
> attribute: Sat, 22 Jun 2019 17:13:32 GMT
> 2019/06/22 10:13:32:458 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: t=NoGNAN4y5h84gjei3tzmqbGl;
> Domain=dropbox.com; expires=Tue, 21 Jun 2022 17:13:32 GMT; httponly;
> Path=/; secure". Invalid 'expires' attribute: Tue, 21 Jun 2022
> 17:13:32 GMT
> 2019/06/22 10:13:32:458 PDT [WARN] ResponseProcessCookies - Invalid
> cookie header: "Set-Cookie: __Host-js_csrf=NoGNAN4y5h84gjei3tzmqbGl;
> expires=Tue, 21 Jun 2022 17:13:32 GMT; Path=/; secure". Invalid
> 'expires' attribute: Tue, 21 Jun 2022 17:13:32 GMT
>
>
>
>
> On Friday, June 21, 2019, 9:03:41 AM PDT, Oleg Kalnichevski <
> olegk@apache.org> wrote:
>
> On Fri, 2019-06-21 at 00:04 +0000, Krishna Sankaran wrote:
> > I am using the following java snippet for a HEAD request. The
> > request
> > gets redirected couple of times, a 301 followed by 302 and dinally
> > gets a 200.When i use the following code snippet, i get the status
> > as
> > 200 OK, however all the header fields are from the 1st response
> > (301). The Content-Length header is present in the last response,
> > but
> > i don't see that in the header's received by the code. Is there a
> > knob to get the last header or all headers?
> >
> > private static final String SAMPLE_URL = "
> > https://www.dropbox.com/s/<something>/test_out4.mp4";
> >
> > public static void main(String[] args) throws IOException,
> > URISyntaxException {
> > HttpClient client = HttpClientBuilder.create().build();
> > HttpHead request = new HttpHead(new URI(SAMPLE_URL));
> > HttpResponse response = client.execute(request);
> > System.out.println(response.getStatusLine());
> > for (Header header : response.getAllHeaders()) {
> > System.out.println(header.getName() + ": " +
> > header.getValue());
> > }
> > }
> >
> > It returns the code as 200 OK, but the Content-Type is incorrect,
> > even
> > the
> > HTTP/1.1 200 OK <<< Status is 200 Server: nginx Date: Thu, 20 Jun
> > 2019 02:22:58 GMT Content-Type: text/html; charset=utf-8 << Content
> > type is char <<<< Content-
> > Length
> > is missing!!!!!
> >
> > curl -I https://www.dropbox.com/s//test_out4.mp4 << Curl works
> > correctly
> >
> >
> >
> > HTTP/1.1 301 Moved Permanently <<< Status 301 Server: nginx
> > Date: Thu, 20 Jun 2019 02:20:50 GMT Content-Type: text/html;
> > charset=utf-8 <<< Content type text Connection: keep-alive
> > ....
> > HTTP/1.1 302 Found << second redirect Server: nginx
> > Date: Thu, 20 Jun 2019 02:36:03 GMT
> > Content-Type: text/html; charset=utf-8
> > ....
> > HTTP/1.1 200 OK <<< Status finally 200 Server: nginx
> > Date: Thu, 20 Jun 2019 02:36:04 GMT
> > Content-Type: video/mp4 << content type correct
> > Content-Length: 92894175 << length correct Connection: keep-alive
> >
> >
>
> Krishna
>
> I _suppose_ this is due automatic content compression used by
> HttpClient per default. This can be easy to tell if you turn on
> HttpClient context / wire logging.
>
> Try disabling automatic content compression.
>
> ---
> CloseableHttpClient client = HttpClients.custom()
> .disableContentCompression()
> .build();
> ---
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HTTP headers for Redirected HEAD requests
Posted by Krishna Sankaran <kr...@yahoo.com.INVALID>.
No, Disabling header compression did not work. One thing to notice is this redirect is not to another host, but a different path in the same server. Does it make a difference? I have attached the log below.
I got the log using the following parameters. I don't see multiple get requests going out. I only one see request/response. even though when i use curl, i see a 301, followed by 302 followed by 200. Also the context.getRedirectLocations() also returns a null.
-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog
-Dorg.apache.commons.logging.simplelog.showdatetime=true
-Dorg.apache.commons.logging.simplelog.log.org.apache.http=DEBUG
-Dorg.apache.commons.logging.simplelog.log.org.apache.http.wire=DEBUG
-Dorg.apache.commons.logging.simplelog.log.org.apache.http.impl.conn=DEBUG
-Dorg.apache.commons.logging.simplelog.log.org.apache.http.client=DEBUG
2019/06/22 10:13:31:796 PDT [DEBUG] RequestAddCookies - CookieSpec selected: default
2019/06/22 10:13:31:818 PDT [DEBUG] RequestAuthCache - Auth cache not set in the context
2019/06/22 10:13:31:820 PDT [DEBUG] PoolingHttpClientConnectionManager - Connection request: [route: {s}->https://www.dropbox.com:443][total kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 20]
2019/06/22 10:13:31:838 PDT [DEBUG] PoolingHttpClientConnectionManager - Connection leased: [id: 0][route: {s}->https://www.dropbox.com:443][total kept alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
2019/06/22 10:13:31:840 PDT [DEBUG] MainClientExec - Opening connection {s}->https://www.dropbox.com:443
2019/06/22 10:13:31:866 PDT [DEBUG] DefaultHttpClientConnectionOperator - Connecting to www.dropbox.com/162.125.7.1:443
2019/06/22 10:13:31:866 PDT [DEBUG] SSLConnectionSocketFactory - Connecting socket to www.dropbox.com/162.125.7.1:443 with timeout 0
2019/06/22 10:13:31:946 PDT [DEBUG] SSLConnectionSocketFactory - Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
2019/06/22 10:13:31:947 PDT [DEBUG] SSLConnectionSocketFactory - Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
2019/06/22 10:13:31:947 PDT [DEBUG] SSLConnectionSocketFactory - Starting handshake
2019/06/22 10:13:32:044 PDT [DEBUG] SSLConnectionSocketFactory - Secure session established
2019/06/22 10:13:32:045 PDT [DEBUG] SSLConnectionSocketFactory - negotiated protocol: TLSv1.2
2019/06/22 10:13:32:045 PDT [DEBUG] SSLConnectionSocketFactory - negotiated cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory - peer principal: CN=www.dropbox.com, O="Dropbox, Inc", L=San Francisco, ST=California, C=US, SERIALNUMBER=4348296, OID.1.3.6.1.4.1.311.60.2.1.2=Delaware, OID.1.3.6.1.4.1.311.60.2.1.3=US, OID.2.5.4.15=Private Organization
2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory - peer alternative names: [www.dropbox.com, dropbox.com, support.dropbox.com, live.dropbox.com, opensource.dropbox.com, linux.dropbox.com, texter.dropbox.com]
2019/06/22 10:13:32:046 PDT [DEBUG] SSLConnectionSocketFactory - issuer principal: CN=DigiCert SHA2 Extended Validation Server CA, OU=www.digicert.com, O=DigiCert Inc, C=US
2019/06/22 10:13:32:050 PDT [DEBUG] DefaultHttpClientConnectionOperator - Connection established 192.168.1.7:50471<->162.125.7.1:443
2019/06/22 10:13:32:050 PDT [DEBUG] MainClientExec - Executing request HEAD /s/<somedir>/test_out4.mp4 HTTP/1.1
2019/06/22 10:13:32:050 PDT [DEBUG] MainClientExec - Target auth state: UNCHALLENGED
2019/06/22 10:13:32:051 PDT [DEBUG] MainClientExec - Proxy auth state: UNCHALLENGED
2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >> HEAD /s/<Somedire>/test_out4.mp4 HTTP/1.1
2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >> User-Agent: MySuperUserAgent
2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >> Host: www.dropbox.com
2019/06/22 10:13:32:053 PDT [DEBUG] headers - http-outgoing-0 >> Connection: Keep-Alive
2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >> "HEAD /s/<somedir>/test_out4.mp4 HTTP/1.1[\r][\n]"
2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >> "User-Agent: MySuperUserAgent[\r][\n]"
2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >> "Host: www.dropbox.com[\r][\n]"
2019/06/22 10:13:32:053 PDT [DEBUG] wire - http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
2019/06/22 10:13:32:054 PDT [DEBUG] wire - http-outgoing-0 >> "[\r][\n]"
2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 << "HTTP/1.1 200 OK[\r][\n]"
2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 << "Server: nginx[\r][\n]"
2019/06/22 10:13:32:429 PDT [DEBUG] wire - http-outgoing-0 << "Date: Sat, 22 Jun 2019 17:13:32 GMT[\r][\n]"
2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Content-Type: text/html; charset=utf-8[\r][\n]"
2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Connection: keep-alive[\r][\n]"
2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Vary: Accept-Encoding[\r][\n]"
2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Vary: Accept-Encoding[\r][\n]"
2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Cache-Control: no-cache[\r][\n]"
2019/06/22 10:13:32:430 PDT [DEBUG] wire - http-outgoing-0 << "Content-Security-Policy: script-src 'unsafe-eval' https://www.dropbox.com/static/compiled/js/ https://www.dropbox.com/static/api/ https://www.dropbox.com/page_success/ https://cfl.dropboxstatic.com/static/compiled/js/ https://www.dropboxstatic.com/static/compiled/js/ https://cfl.dropboxstatic.com/static/js/ https://www.dropboxstatic.com/static/js/ https://cfl.dropboxstatic.com/static/src/dws-ensemble-appshell/ https://www.dropboxstatic.com/static/src/dws-ensemble-appshell/ https://cfl.dropboxstatic.com/static/previews/ https://www.dropboxstatic.com/static/previews/ https://cfl.dropboxstatic.com/static/api/ https://www.dropboxstatic.com/static/api/ https://cfl.dropboxstatic.com/static/cms/ https://www.dropboxstatic.com/static/cms/ https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ 'unsafe-inline' ; img-src https://* data: blob: ; frame-ancestors 'self' ; default-src https://www.dropbox.com/playlist/ https://www.dropbox.com/v/s/playlist/ https://*.dropboxusercontent.com/p/hls_master_playlist/ https://*.dropboxusercontent.com/p/hls_playlist/ ; frame-src https://* carousel://* dbapi-6://* dbapi-7://* dbapi-8://* itms-apps://* itms-appss://* ; worker-src https://www.dropbox.com/static/serviceworker/ blob: ; style-src https://* 'unsafe-inline' 'unsafe-eval' ; connect-src https://* ws://127.0.0.1:*/ws ; object-src 'self' https://cfl.dropboxstatic.com/static/ https://www.dropboxstatic.com/static/ https://flash.dropboxstatic.com https://swf.dropboxstatic.com https://dbxlocal.dropboxstatic.com ; media-src https://* blob: ; font-src https://* data: ; child-src https://www.dropbox.com/static/serviceworker/ blob: ; form-action 'self' https://www.dropbox.com/ https://dl-web.dropbox.com/ https://photos.dropbox.com/ https://paper.dropbox.com/ https://showcase.dropbox.com/ https://accounts.google.com/ https://api.login.yahoo.com/ https://www.hellofax.com/ https://app.hellofax.com/ https://www.hellosign.com/ https://app.hellosign.com/ https://login.yahoo.com/ ; base-uri 'self' ; report-uri https://www.dropbox.com/csp_log?policy_name=metaserver-whitelist[\r][\n]"
2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Dropbox-Streaming: V=1[\r][\n]"
2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Pragma: no-cache[\r][\n]"
2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Referrer-Policy: origin-when-cross-origin[\r][\n]"
2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D; expires=Mon, 22 Jul 2019 17:13:32 GMT; Path=/; secure[\r][\n]"
2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-Cookie: locale=en; Domain=dropbox.com; expires=Thu, 20 Jun 2024 17:13:32 GMT; Path=/; secure[\r][\n]"
2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-Cookie: gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D; expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/; secure[\r][\n]"
2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-Cookie: flash=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32 GMT; Path=/; secure[\r][\n]"
2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32 GMT; httponly; Path=/; secure[\r][\n]"
2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-Cookie: bang=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32 GMT; Path=/; secure[\r][\n]"
2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-Cookie: t=NoGNAN4y5h84gjei3tzmqbGl; Domain=dropbox.com; expires=Tue, 21 Jun 2022 17:13:32 GMT; httponly; Path=/; secure[\r][\n]"
2019/06/22 10:13:32:431 PDT [DEBUG] wire - http-outgoing-0 << "Set-Cookie: __Host-js_csrf=NoGNAN4y5h84gjei3tzmqbGl; expires=Tue, 21 Jun 2022 17:13:32 GMT; Path=/; secure[\r][\n]"
2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-Content-Type-Options: nosniff[\r][\n]"
2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-Dropbox-Request-Id: 2d37a02490831955c27085616282ef96[\r][\n]"
2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-Frame-Options: DENY[\r][\n]"
2019/06/22 10:13:32:432 PDT [DEBUG] wire - http-outgoing-0 << "X-Robots-Tag: noindex, nofollow, noimageindex[\r][\n]"
2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 << "X-Xss-Protection: 1; mode=block[\r][\n]"
2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 << "Strict-Transport-Security: max-age=15552000; includeSubDomains[\r][\n]"
2019/06/22 10:13:32:434 PDT [DEBUG] wire - http-outgoing-0 << "[\r][\n]"
2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 << HTTP/1.1 200 OK
2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 << Server: nginx
2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 << Date: Sat, 22 Jun 2019 17:13:32 GMT
2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 << Content-Type: text/html; charset=utf-8
2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 << Connection: keep-alive
2019/06/22 10:13:32:438 PDT [DEBUG] headers - http-outgoing-0 << Vary: Accept-Encoding
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Vary: Accept-Encoding
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Cache-Control: no-cache
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Content-Security-Policy: script-src 'unsafe-eval' https://www.dropbox.com/static/compiled/js/ https://www.dropbox.com/static/api/ https://www.dropbox.com/page_success/ https://cfl.dropboxstatic.com/static/compiled/js/ https://www.dropboxstatic.com/static/compiled/js/ https://cfl.dropboxstatic.com/static/js/ https://www.dropboxstatic.com/static/js/ https://cfl.dropboxstatic.com/static/src/dws-ensemble-appshell/ https://www.dropboxstatic.com/static/src/dws-ensemble-appshell/ https://cfl.dropboxstatic.com/static/previews/ https://www.dropboxstatic.com/static/previews/ https://cfl.dropboxstatic.com/static/api/ https://www.dropboxstatic.com/static/api/ https://cfl.dropboxstatic.com/static/cms/ https://www.dropboxstatic.com/static/cms/ https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ 'unsafe-inline' ; img-src https://* data: blob: ; frame-ancestors 'self' ; default-src https://www.dropbox.com/playlist/ https://www.dropbox.com/v/s/playlist/ https://*.dropboxusercontent.com/p/hls_master_playlist/ https://*.dropboxusercontent.com/p/hls_playlist/ ; frame-src https://* carousel://* dbapi-6://* dbapi-7://* dbapi-8://* itms-apps://* itms-appss://* ; worker-src https://www.dropbox.com/static/serviceworker/ blob: ; style-src https://* 'unsafe-inline' 'unsafe-eval' ; connect-src https://* ws://127.0.0.1:*/ws ; object-src 'self' https://cfl.dropboxstatic.com/static/ https://www.dropboxstatic.com/static/ https://flash.dropboxstatic.com https://swf.dropboxstatic.com https://dbxlocal.dropboxstatic.com ; media-src https://* blob: ; font-src https://* data: ; child-src https://www.dropbox.com/static/serviceworker/ blob: ; form-action 'self' https://www.dropbox.com/ https://dl-web.dropbox.com/ https://photos.dropbox.com/ https://paper.dropbox.com/ https://showcase.dropbox.com/ https://accounts.google.com/ https://api.login.yahoo.com/ https://www.hellofax.com/ https://app.hellofax.com/ https://www.hellosign.com/ https://app.hellosign.com/ https://login.yahoo.com/ ; base-uri 'self' ; report-uri https://www.dropbox.com/csp_log?policy_name=metaserver-whitelist
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Dropbox-Streaming: V=1
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Pragma: no-cache
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Referrer-Policy: origin-when-cross-origin
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D; expires=Mon, 22 Jul 2019 17:13:32 GMT; Path=/; secure
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-Cookie: locale=en; Domain=dropbox.com; expires=Thu, 20 Jun 2024 17:13:32 GMT; Path=/; secure
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-Cookie: gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D; expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/; secure
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-Cookie: flash=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32 GMT; Path=/; secure
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32 GMT; httponly; Path=/; secure
2019/06/22 10:13:32:439 PDT [DEBUG] headers - http-outgoing-0 << Set-Cookie: bang=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32 GMT; Path=/; secure
2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << Set-Cookie: t=NoGNAN4y5h84gjei3tzmqbGl; Domain=dropbox.com; expires=Tue, 21 Jun 2022 17:13:32 GMT; httponly; Path=/; secure
2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << Set-Cookie: __Host-js_csrf=NoGNAN4y5h84gjei3tzmqbGl; expires=Tue, 21 Jun 2022 17:13:32 GMT; Path=/; secure
2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-Content-Type-Options: nosniff
2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-Dropbox-Request-Id: 2d37a02490831955c27085616282ef96
2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-Frame-Options: DENY
2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-Robots-Tag: noindex, nofollow, noimageindex
2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << X-Xss-Protection: 1; mode=block
2019/06/22 10:13:32:440 PDT [DEBUG] headers - http-outgoing-0 << Strict-Transport-Security: max-age=15552000; includeSubDomains
2019/06/22 10:13:32:444 PDT [DEBUG] MainClientExec - Connection can be kept alive indefinitely
2019/06/22 10:13:32:445 PDT [DEBUG] PoolingHttpClientConnectionManager - Connection [id: 0][route: {s}->https://www.dropbox.com:443] can be kept alive indefinitely
2019/06/22 10:13:32:445 PDT [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-0: set socket timeout to 0
2019/06/22 10:13:32:445 PDT [DEBUG] PoolingHttpClientConnectionManager - Connection released: [id: 0][route: {s}->https://www.dropbox.com:443][total kept alive: 1; route allocated: 1 of 2; total allocated: 1 of 20]
2019/06/22 10:13:32:455 PDT [WARN] ResponseProcessCookies - Invalid cookie header: "Set-Cookie: seen-sl-signup-modal=VHJ1ZQ%3D%3D; expires=Mon, 22 Jul 2019 17:13:32 GMT; Path=/; secure". Invalid 'expires' attribute: Mon, 22 Jul 2019 17:13:32 GMT
2019/06/22 10:13:32:456 PDT [WARN] ResponseProcessCookies - Invalid cookie header: "Set-Cookie: locale=en; Domain=dropbox.com; expires=Thu, 20 Jun 2024 17:13:32 GMT; Path=/; secure". Invalid 'expires' attribute: Thu, 20 Jun 2024 17:13:32 GMT
2019/06/22 10:13:32:456 PDT [WARN] ResponseProcessCookies - Invalid cookie header: "Set-Cookie: gvc=Mzk5NDUxMzQzNjY3NzkzNDkwMTA4NTY4NzMxNzA3MDMyNzYzNDY%3D; expires=Thu, 20 Jun 2024 17:13:32 GMT; httponly; Path=/; secure". Invalid 'expires' attribute: Thu, 20 Jun 2024 17:13:32 GMT
2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid cookie header: "Set-Cookie: flash=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32 GMT; Path=/; secure". Invalid 'expires' attribute: Sat, 22 Jun 2019 17:13:32 GMT
2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid cookie header: "Set-Cookie: puc=; expires=Sat, 22 Jun 2019 17:13:32 GMT; httponly; Path=/; secure". Invalid 'expires' attribute: Sat, 22 Jun 2019 17:13:32 GMT
2019/06/22 10:13:32:457 PDT [WARN] ResponseProcessCookies - Invalid cookie header: "Set-Cookie: bang=; Domain=dropbox.com; expires=Sat, 22 Jun 2019 17:13:32 GMT; Path=/; secure". Invalid 'expires' attribute: Sat, 22 Jun 2019 17:13:32 GMT
2019/06/22 10:13:32:458 PDT [WARN] ResponseProcessCookies - Invalid cookie header: "Set-Cookie: t=NoGNAN4y5h84gjei3tzmqbGl; Domain=dropbox.com; expires=Tue, 21 Jun 2022 17:13:32 GMT; httponly; Path=/; secure". Invalid 'expires' attribute: Tue, 21 Jun 2022 17:13:32 GMT
2019/06/22 10:13:32:458 PDT [WARN] ResponseProcessCookies - Invalid cookie header: "Set-Cookie: __Host-js_csrf=NoGNAN4y5h84gjei3tzmqbGl; expires=Tue, 21 Jun 2022 17:13:32 GMT; Path=/; secure". Invalid 'expires' attribute: Tue, 21 Jun 2022 17:13:32 GMT
On Friday, June 21, 2019, 9:03:41 AM PDT, Oleg Kalnichevski <ol...@apache.org> wrote:
On Fri, 2019-06-21 at 00:04 +0000, Krishna Sankaran wrote:
> I am using the following java snippet for a HEAD request. The request
> gets redirected couple of times, a 301 followed by 302 and dinally
> gets a 200.When i use the following code snippet, i get the status as
> 200 OK, however all the header fields are from the 1st response
> (301). The Content-Length header is present in the last response, but
> i don't see that in the header's received by the code. Is there a
> knob to get the last header or all headers?
>
> private static final String SAMPLE_URL = "
> https://www.dropbox.com/s/<something>/test_out4.mp4";
>
> public static void main(String[] args) throws IOException,
> URISyntaxException {
> HttpClient client = HttpClientBuilder.create().build();
> HttpHead request = new HttpHead(new URI(SAMPLE_URL));
> HttpResponse response = client.execute(request);
> System.out.println(response.getStatusLine());
> for (Header header : response.getAllHeaders()) {
> System.out.println(header.getName() + ": " +
> header.getValue());
> }
> }
>
> It returns the code as 200 OK, but the Content-Type is incorrect,
> even
> the
> HTTP/1.1 200 OK <<< Status is 200 Server: nginx Date: Thu, 20 Jun
> 2019 02:22:58 GMT Content-Type: text/html; charset=utf-8 << Content
> type is char <<<< Content-Length
> is missing!!!!!
>
> curl -I https://www.dropbox.com/s//test_out4.mp4 << Curl works
> correctly
>
>
>
> HTTP/1.1 301 Moved Permanently <<< Status 301 Server: nginx
> Date: Thu, 20 Jun 2019 02:20:50 GMT Content-Type: text/html;
> charset=utf-8 <<< Content type text Connection: keep-alive
> ....
> HTTP/1.1 302 Found << second redirect Server: nginx
> Date: Thu, 20 Jun 2019 02:36:03 GMT
> Content-Type: text/html; charset=utf-8
> ....
> HTTP/1.1 200 OK <<< Status finally 200 Server: nginx
> Date: Thu, 20 Jun 2019 02:36:04 GMT
> Content-Type: video/mp4 << content type correct
> Content-Length: 92894175 << length correct Connection: keep-alive
>
>
Krishna
I _suppose_ this is due automatic content compression used by
HttpClient per default. This can be easy to tell if you turn on
HttpClient context / wire logging.
Try disabling automatic content compression.
---
CloseableHttpClient client = HttpClients.custom()
.disableContentCompression()
.build();
---
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HTTP headers for Redirected HEAD requests
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Fri, 2019-06-21 at 00:04 +0000, Krishna Sankaran wrote:
> I am using the following java snippet for a HEAD request. The request
> gets redirected couple of times, a 301 followed by 302 and dinally
> gets a 200.When i use the following code snippet, i get the status as
> 200 OK, however all the header fields are from the 1st response
> (301). The Content-Length header is present in the last response, but
> i don't see that in the header's received by the code. Is there a
> knob to get the last header or all headers?
>
> private static final String SAMPLE_URL = "
> https://www.dropbox.com/s/<something>/test_out4.mp4";
>
> public static void main(String[] args) throws IOException,
> URISyntaxException {
> HttpClient client = HttpClientBuilder.create().build();
> HttpHead request = new HttpHead(new URI(SAMPLE_URL));
> HttpResponse response = client.execute(request);
> System.out.println(response.getStatusLine());
> for (Header header : response.getAllHeaders()) {
> System.out.println(header.getName() + ": " +
> header.getValue());
> }
> }
>
> It returns the code as 200 OK, but the Content-Type is incorrect,
> even
> the
> HTTP/1.1 200 OK <<< Status is 200 Server: nginx Date: Thu, 20 Jun
> 2019 02:22:58 GMT Content-Type: text/html; charset=utf-8 << Content
> type is char <<<< Content-Length
> is missing!!!!!
>
> curl -I https://www.dropbox.com/s//test_out4.mp4 << Curl works
> correctly
>
>
>
> HTTP/1.1 301 Moved Permanently <<< Status 301 Server: nginx
> Date: Thu, 20 Jun 2019 02:20:50 GMT Content-Type: text/html;
> charset=utf-8 <<< Content type text Connection: keep-alive
> ....
> HTTP/1.1 302 Found << second redirect Server: nginx
> Date: Thu, 20 Jun 2019 02:36:03 GMT
> Content-Type: text/html; charset=utf-8
> ....
> HTTP/1.1 200 OK <<< Status finally 200 Server: nginx
> Date: Thu, 20 Jun 2019 02:36:04 GMT
> Content-Type: video/mp4 << content type correct
> Content-Length: 92894175 << length correct Connection: keep-alive
>
>
Krishna
I _suppose_ this is due automatic content compression used by
HttpClient per default. This can be easy to tell if you turn on
HttpClient context / wire logging.
Try disabling automatic content compression.
---
CloseableHttpClient client = HttpClients.custom()
.disableContentCompression()
.build();
---
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org