You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Claudio Martella <cl...@tis.bz.it> on 2011/01/05 14:08:44 UTC
httpclient 3.1 failing DIGEST authentication
Hello list,
I'm using Apache Nutch to crawl my intranet which is under DIGEST
authentication (nutch is using httpclient 3.1).
As the client is failing the auth with the same credentials my browser
is succeeding, I wrote an example app to try figure out what's going wrong.
Here it is:
HttpClient client = new HttpClient();
client.getParams().setAuthenticationPreemptive(true);
Credentials defaultcreds = new
UsernamePasswordCredentials("user", "*******");
List authPrefs = new ArrayList();
authPrefs.add(AuthPolicy.DIGEST);
client.getParams().setParameter(AuthPolicy.AUTH_SCHEME_PRIORITY,
authPrefs);
client.getState().setCredentials(AuthScope.ANY, defaultcreds);
HttpMethod method = new GetMethod("http://192.168.10.209:8090");
What I can see from the logs is that the client is trying to
authenticate with Basic authentication but the server expects NTLM and
only NTLM. Am I reading it correctly?
Why isn't it trying to authenticate with Digest as requested?
Here are the logs:
2011/01/05 13:25:07:566 CET [DEBUG] HttpClient - Java version: 1.6.0_22
2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Java vendor: Apple Inc.
2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Java class path:
/Users/hammer/TIS/java-hacking/auth-test/target/classes:/Users/hammer/.m2/repository/commons-codec/commons-codec/1.2/commons-codec-1.2.jar:/Users/hammer/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/Users/hammer/.m2/repository/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.jar:/Users/hammer/.m2/repository/org/apache/httpcomponents/httpclient/4.0.3/httpclient-4.0.3.jar:/Users/hammer/.m2/repository/org/apache/httpcomponents/httpcore/4.0.1/httpcore-4.0.1.jar:/Users/hammer/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar:/Users/hammer/.m2/repository/log4j/log4j/1.2.14/log4j-1.2.14.jar
2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Operating system name:
Mac OS X
2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Operating system
architecture: x86_64
2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Operating system
version: 10.5.8
2011/01/05 13:25:07:697 CET [DEBUG] HttpClient - SUN 1.6: SUN (DSA
key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom;
X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
CertPathBuilder; LDAP, Collection CertStores, JavaPolicy Policy;
JavaLoginConfig Configuration)
2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - Apple 1.0: Apple
Provider (implements DES, Triple DES, AES, Blowfish, PBE,
Diffie-Hellman, HMAC/MD5, HMAC/SHA1)
2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunRsaSign 1.5: Sun RSA
signature provider
2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunJSSE 1.6: Sun JSSE
provider(PKCS12, SunX509 key/trust factories, SSLv3, TLSv1)
2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunJCE 1.6: SunJCE
Provider (implements RSA, DES, Triple DES, AES, Blowfish, ARCFOUR, RC2,
PBE, Diffie-Hellman, HMAC)
2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunJGSS 1.0: Sun
(Kerberos v5, SPNEGO)
2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunSASL 1.5: Sun SASL
provider(implements client mechanisms for: DIGEST-MD5, GSSAPI, EXTERNAL,
PLAIN, CRAM-MD5; server mechanisms for: DIGEST-MD5, GSSAPI, CRAM-MD5)
2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - XMLDSig 1.0: XMLDSig
(DOM XMLSignatureFactory; DOM KeyInfoFactory)
2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunPCSC 1.6: Sun PC/SC
provider
2011/01/05 13:25:07:703 CET [DEBUG] DefaultHttpParams - Set parameter
http.useragent = Jakarta Commons-HttpClient/3.1
2011/01/05 13:25:07:705 CET [DEBUG] DefaultHttpParams - Set parameter
http.protocol.version = HTTP/1.1
2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
http.connection-manager.class = class
org.apache.commons.httpclient.SimpleHttpConnectionManager
2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
http.protocol.cookie-policy = default
2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
http.protocol.element-charset = US-ASCII
2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
http.protocol.content-charset = ISO-8859-1
2011/01/05 13:25:07:708 CET [DEBUG] DefaultHttpParams - Set parameter
http.method.retry-handler =
org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@41fc2fb
2011/01/05 13:25:07:708 CET [DEBUG] DefaultHttpParams - Set parameter
http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE,
dd-MMM-yy HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy
HH:mm:ss z, EEE, dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE
dd-MMM-yyyy HH:mm:ss z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy
HH-mm-ss z, EEE dd-MMM-yy HH:mm:ss z, EEE dd MMM yy HH:mm:ss z,
EEE,dd-MMM-yy HH:mm:ss z, EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy
HH:mm:ss z]
2011/01/05 13:25:07:713 CET [DEBUG] DefaultHttpParams - Set parameter
http.authentication.preemptive = true
2011/01/05 13:25:07:715 CET [DEBUG] DefaultHttpParams - Set parameter
http.auth.scheme-priority = [Digest]
2011/01/05 13:25:07:764 CET [DEBUG] DefaultHttpParams - Set parameter
http.method.retry-handler =
org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@450e790c
2011/01/05 13:25:07:774 CET [DEBUG] HttpMethodDirector - Preemptively
sending default basic credentials
2011/01/05 13:25:07:783 CET [DEBUG] HttpMethodDirector - Authenticating
with BASIC <any realm>@192.168.10.210:8090
2011/01/05 13:25:07:783 CET [DEBUG] HttpMethodParams - Credential
charset not configured, using HTTP element charset
2011/01/05 13:25:07:785 CET [DEBUG] HttpConnection - Open connection to
192.168.10.210:8090
2011/01/05 13:25:07:818 CET [DEBUG] header - >> "GET / HTTP/1.1[\r][\n]"
2011/01/05 13:25:07:819 CET [DEBUG] HttpMethodBase - Adding Host request
header
2011/01/05 13:25:07:832 CET [DEBUG] header - >> "Authorization: Basic
****************************[\r][\n]"
2011/01/05 13:25:07:832 CET [DEBUG] header - >> "User-Agent: Jakarta
Commons-HttpClient/3.1[\r][\n]"
2011/01/05 13:25:07:833 CET [DEBUG] header - >> "Host:
192.168.10.210:8090[\r][\n]"
2011/01/05 13:25:07:833 CET [DEBUG] header - >> "[\r][\n]"
2011/01/05 13:25:07:835 CET [DEBUG] header - << "HTTP/1.1 401
Unauthorized[\r][\n]"
2011/01/05 13:25:07:835 CET [DEBUG] header - << "HTTP/1.1 401
Unauthorized[\r][\n]"
2011/01/05 13:25:07:836 CET [DEBUG] header - << "Content-Length:
1656[\r][\n]"
2011/01/05 13:25:07:836 CET [DEBUG] header - << "Content-Type:
text/html[\r][\n]"
2011/01/05 13:25:07:836 CET [DEBUG] header - << "Server:
Microsoft-IIS/6.0[\r][\n]"
2011/01/05 13:25:07:837 CET [DEBUG] header - << "WWW-Authenticate:
Negotiate[\r][\n]"
2011/01/05 13:25:07:837 CET [DEBUG] header - << "WWW-Authenticate:
NTLM[\r][\n]"
2011/01/05 13:25:07:837 CET [DEBUG] header - << "X-Powered-By:
ASP.NET[\r][\n]"
2011/01/05 13:25:07:837 CET [DEBUG] header - << "Date: Wed, 05 Jan 2011
12:25:07 GMT[\r][\n]"
2011/01/05 13:25:07:837 CET [DEBUG] header - << "[\r][\n]"
2011/01/05 13:25:07:838 CET [DEBUG] HttpMethodDirector - Authorization
required
2011/01/05 13:25:07:839 CET [DEBUG] AuthChallengeProcessor - Supported
authentication schemes in the order of preference: [Digest]
2011/01/05 13:25:07:839 CET [DEBUG] AuthChallengeProcessor - Challenge
for Digest authentication scheme not available
2011/01/05 13:25:07:840 CET [WARN] HttpMethodDirector - Unable to
respond to any of these challenges: {ntlm=NTLM, negotiate=Negotiate}
Method failed: HTTP/1.1 401 Unauthorized
2011/01/05 13:25:07:840 CET [DEBUG] HttpMethodBase - Buffering response body
2011/01/05 13:25:07:841 CET [DEBUG] HttpMethodBase - Resorting to
protocol version default close connection policy
2011/01/05 13:25:07:841 CET [DEBUG] HttpMethodBase - Should NOT close
connection, using HTTP/1.1
2011/01/05 13:25:07:841 CET [DEBUG] HttpConnection - Releasing
connection back to connection manager.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>You are not authorized to view this page</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=Windows-1252">
<STYLE type="text/css">
BODY { font: 8pt/12pt verdana }
H1 { font: 13pt/15pt verdana }
H2 { font: 8pt/12pt verdana }
A:link { color: red }
A:visited { color: maroon }
</STYLE>
</HEAD><BODY><TABLE width=500 border=0 cellspacing=10><TR><TD>
<h1>You are not authorized to view this page</h1>
You do not have permission to view this directory or page using the
credentials that you supplied because your Web browser is sending a
WWW-Authenticate header field that the Web server is not configured to
accept.
<hr>
<p>Please try the following:</p>
<ul>
<li>Contact the Web site administrator if you believe you should be able
to view this directory or page.</li>
<li>Click the <a href="javascript:location.reload()">Refresh</a> button
to try again with different credentials.</li>
</ul>
<h2>HTTP Error 401.2 - Unauthorized: Access is denied due to server
configuration.<br>Internet Information Services (IIS)</h2>
<hr>
<p>Technical Information (for support personnel)</p>
<ul>
<li>Go to <a
href="http://go.microsoft.com/fwlink/?linkid=8180">Microsoft Product
Support Services</a> and perform a title search for the words
<b>HTTP</b> and <b>401</b>.</li>
<li>Open <b>IIS Help</b>, which is accessible in IIS Manager (inetmgr),
and search for topics titled <b>About Security</b>,
<b>Authentication</b>, and <b>About Custom Error Messages</b>.</li>
</ul>
</TD></TR></TABLE></BODY></HTML>
--
Claudio Martella
Digital Technologies
Unit Research & Development - Analyst
TIS innovation park
Via Siemens 19 | Siemensstr. 19
39100 Bolzano | 39100 Bozen
Tel. +39 0471 068 123
Fax +39 0471 068 129
claudio.martella@tis.bz.it http://www.tis.bz.it
Short information regarding use of personal data. According to Section 13 of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we process your personal data in order to fulfil contractual and fiscal obligations and also to send you information regarding our services and events. Your personal data are processed with and without electronic means and by respecting data subjects' rights, fundamental freedoms and dignity, particularly with regard to confidentiality, personal identity and the right to personal data protection. At any time and without formalities you can write an e-mail to privacy@tis.bz.it in order to object the processing of your personal data for the purpose of sending advertising materials and also to exercise the right to access personal data and other rights referred to in Section 7 of Decree 196/2003. The data controller is TIS Techno Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the complete information on the web site www.tis.bz.it.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: httpclient 3.1 failing DIGEST authentication
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2011-01-05 at 15:54 +0100, Claudio Martella wrote:
> Done.
>
> It basically does a simple Basic authentication putting user and
> password in the request:
>
> http://user:password@ip/
>
> This doesn't happen with httpclient which tries a basic authentication
> through the Authenticate header entry.
> The question, though, is why although configured like i did, httpclient
> does Basic instead of Digest like it should.
>
HttpClient 3.1 is not capable of using any other scheme for preemptive
authentication but BASIC. Besides, it is pointless to force DIGEST
authentication in this case as the server would accept NTLM only.
HttpClient 3.1 is EOL and is no longer supported. You may want to raise
this as a change request with developers of Nutch and ask them to
consider upgrading to HttpClient 4.x or 4.1.x
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: httpclient 3.1 failing DIGEST authentication
Posted by Claudio Martella <cl...@tis.bz.it>.
Done.
It basically does a simple Basic authentication putting user and
password in the request:
http://user:password@ip/
This doesn't happen with httpclient which tries a basic authentication
through the Authenticate header entry.
The question, though, is why although configured like i did, httpclient
does Basic instead of Digest like it should.
On 1/5/11 3:32 PM, Ryan Smith wrote:
> You can log the browser session and then compare the working browser session
> header log with the httpClient header log below and see which headers are
> different. You can use tcpflow or wireshark to capture the http traffic
> from your browser. Firefox also has some plugins for logging http headers
> to compare with your httpClient log below. Im not too familiar with
> httpClient auth, but this will show you what httpClient headers are
> different from your browser's headers.
>
> hth.
>
>
> On Wed, Jan 5, 2011 at 8:08 AM, Claudio Martella <claudio.martella@tis.bz.it
>> wrote:
>> Hello list,
>>
>> I'm using Apache Nutch to crawl my intranet which is under DIGEST
>> authentication (nutch is using httpclient 3.1).
>> As the client is failing the auth with the same credentials my browser
>> is succeeding, I wrote an example app to try figure out what's going wrong.
>>
>> Here it is:
>>
>> HttpClient client = new HttpClient();
>> client.getParams().setAuthenticationPreemptive(true);
>> Credentials defaultcreds = new
>> UsernamePasswordCredentials("user", "*******");
>> List authPrefs = new ArrayList();
>> authPrefs.add(AuthPolicy.DIGEST);
>> client.getParams().setParameter(AuthPolicy.AUTH_SCHEME_PRIORITY,
>> authPrefs);
>> client.getState().setCredentials(AuthScope.ANY, defaultcreds);
>> HttpMethod method = new GetMethod("http://192.168.10.209:8090");
>>
>>
>> What I can see from the logs is that the client is trying to
>> authenticate with Basic authentication but the server expects NTLM and
>> only NTLM. Am I reading it correctly?
>> Why isn't it trying to authenticate with Digest as requested?
>>
>>
>> Here are the logs:
>>
>> 2011/01/05 13:25:07:566 CET [DEBUG] HttpClient - Java version: 1.6.0_22
>> 2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Java vendor: Apple Inc.
>> 2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Java class path:
>>
>> /Users/hammer/TIS/java-hacking/auth-test/target/classes:/Users/hammer/.m2/repository/commons-codec/commons-codec/1.2/commons-codec-1.2.jar:/Users/hammer/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/Users/hammer/.m2/repository/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.jar:/Users/hammer/.m2/repository/org/apache/httpcomponents/httpclient/4.0.3/httpclient-4.0.3.jar:/Users/hammer/.m2/repository/org/apache/httpcomponents/httpcore/4.0.1/httpcore-4.0.1.jar:/Users/hammer/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar:/Users/hammer/.m2/repository/log4j/log4j/1.2.14/log4j-1.2.14.jar
>> 2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Operating system name:
>> Mac OS X
>> 2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Operating system
>> architecture: x86_64
>> 2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Operating system
>> version: 10.5.8
>> 2011/01/05 13:25:07:697 CET [DEBUG] HttpClient - SUN 1.6: SUN (DSA
>> key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom;
>> X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
>> CertPathBuilder; LDAP, Collection CertStores, JavaPolicy Policy;
>> JavaLoginConfig Configuration)
>> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - Apple 1.0: Apple
>> Provider (implements DES, Triple DES, AES, Blowfish, PBE,
>> Diffie-Hellman, HMAC/MD5, HMAC/SHA1)
>> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunRsaSign 1.5: Sun RSA
>> signature provider
>> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunJSSE 1.6: Sun JSSE
>> provider(PKCS12, SunX509 key/trust factories, SSLv3, TLSv1)
>> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunJCE 1.6: SunJCE
>> Provider (implements RSA, DES, Triple DES, AES, Blowfish, ARCFOUR, RC2,
>> PBE, Diffie-Hellman, HMAC)
>> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunJGSS 1.0: Sun
>> (Kerberos v5, SPNEGO)
>> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunSASL 1.5: Sun SASL
>> provider(implements client mechanisms for: DIGEST-MD5, GSSAPI, EXTERNAL,
>> PLAIN, CRAM-MD5; server mechanisms for: DIGEST-MD5, GSSAPI, CRAM-MD5)
>> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - XMLDSig 1.0: XMLDSig
>> (DOM XMLSignatureFactory; DOM KeyInfoFactory)
>> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunPCSC 1.6: Sun PC/SC
>> provider
>> 2011/01/05 13:25:07:703 CET [DEBUG] DefaultHttpParams - Set parameter
>> http.useragent = Jakarta Commons-HttpClient/3.1
>> 2011/01/05 13:25:07:705 CET [DEBUG] DefaultHttpParams - Set parameter
>> http.protocol.version = HTTP/1.1
>> 2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
>> http.connection-manager.class = class
>> org.apache.commons.httpclient.SimpleHttpConnectionManager
>> 2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
>> http.protocol.cookie-policy = default
>> 2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
>> http.protocol.element-charset = US-ASCII
>> 2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
>> http.protocol.content-charset = ISO-8859-1
>> 2011/01/05 13:25:07:708 CET [DEBUG] DefaultHttpParams - Set parameter
>> http.method.retry-handler =
>> org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@41fc2fb
>> 2011/01/05 13:25:07:708 CET [DEBUG] DefaultHttpParams - Set parameter
>> http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE,
>> dd-MMM-yy HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy
>> HH:mm:ss z, EEE, dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE
>> dd-MMM-yyyy HH:mm:ss z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy
>> HH-mm-ss z, EEE dd-MMM-yy HH:mm:ss z, EEE dd MMM yy HH:mm:ss z,
>> EEE,dd-MMM-yy HH:mm:ss z, EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy
>> HH:mm:ss z]
>> 2011/01/05 13:25:07:713 CET [DEBUG] DefaultHttpParams - Set parameter
>> http.authentication.preemptive = true
>> 2011/01/05 13:25:07:715 CET [DEBUG] DefaultHttpParams - Set parameter
>> http.auth.scheme-priority = [Digest]
>> 2011/01/05 13:25:07:764 CET [DEBUG] DefaultHttpParams - Set parameter
>> http.method.retry-handler =
>> org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@450e790c
>> 2011/01/05 13:25:07:774 CET [DEBUG] HttpMethodDirector - Preemptively
>> sending default basic credentials
>> 2011/01/05 13:25:07:783 CET [DEBUG] HttpMethodDirector - Authenticating
>> with BASIC <any realm>@192.168.10.210:8090
>> 2011/01/05 13:25:07:783 CET [DEBUG] HttpMethodParams - Credential
>> charset not configured, using HTTP element charset
>> 2011/01/05 13:25:07:785 CET [DEBUG] HttpConnection - Open connection to
>> 192.168.10.210:8090
>> 2011/01/05 13:25:07:818 CET [DEBUG] header - >> "GET / HTTP/1.1[\r][\n]"
>> 2011/01/05 13:25:07:819 CET [DEBUG] HttpMethodBase - Adding Host request
>> header
>> 2011/01/05 13:25:07:832 CET [DEBUG] header - >> "Authorization: Basic
>> ****************************[\r][\n]"
>> 2011/01/05 13:25:07:832 CET [DEBUG] header - >> "User-Agent: Jakarta
>> Commons-HttpClient/3.1[\r][\n]"
>> 2011/01/05 13:25:07:833 CET [DEBUG] header - >> "Host:
>> 192.168.10.210:8090[\r][\n]"
>> 2011/01/05 13:25:07:833 CET [DEBUG] header - >> "[\r][\n]"
>> 2011/01/05 13:25:07:835 CET [DEBUG] header - << "HTTP/1.1 401
>> Unauthorized[\r][\n]"
>> 2011/01/05 13:25:07:835 CET [DEBUG] header - << "HTTP/1.1 401
>> Unauthorized[\r][\n]"
>> 2011/01/05 13:25:07:836 CET [DEBUG] header - << "Content-Length:
>> 1656[\r][\n]"
>> 2011/01/05 13:25:07:836 CET [DEBUG] header - << "Content-Type:
>> text/html[\r][\n]"
>> 2011/01/05 13:25:07:836 CET [DEBUG] header - << "Server:
>> Microsoft-IIS/6.0[\r][\n]"
>> 2011/01/05 13:25:07:837 CET [DEBUG] header - << "WWW-Authenticate:
>> Negotiate[\r][\n]"
>> 2011/01/05 13:25:07:837 CET [DEBUG] header - << "WWW-Authenticate:
>> NTLM[\r][\n]"
>> 2011/01/05 13:25:07:837 CET [DEBUG] header - << "X-Powered-By:
>> ASP.NET[\r][\n]"
>> 2011/01/05 13:25:07:837 CET [DEBUG] header - << "Date: Wed, 05 Jan 2011
>> 12:25:07 GMT[\r][\n]"
>> 2011/01/05 13:25:07:837 CET [DEBUG] header - << "[\r][\n]"
>> 2011/01/05 13:25:07:838 CET [DEBUG] HttpMethodDirector - Authorization
>> required
>> 2011/01/05 13:25:07:839 CET [DEBUG] AuthChallengeProcessor - Supported
>> authentication schemes in the order of preference: [Digest]
>> 2011/01/05 13:25:07:839 CET [DEBUG] AuthChallengeProcessor - Challenge
>> for Digest authentication scheme not available
>> 2011/01/05 13:25:07:840 CET [WARN] HttpMethodDirector - Unable to
>> respond to any of these challenges: {ntlm=NTLM, negotiate=Negotiate}
>> Method failed: HTTP/1.1 401 Unauthorized
>> 2011/01/05 13:25:07:840 CET [DEBUG] HttpMethodBase - Buffering response
>> body
>> 2011/01/05 13:25:07:841 CET [DEBUG] HttpMethodBase - Resorting to
>> protocol version default close connection policy
>> 2011/01/05 13:25:07:841 CET [DEBUG] HttpMethodBase - Should NOT close
>> connection, using HTTP/1.1
>> 2011/01/05 13:25:07:841 CET [DEBUG] HttpConnection - Releasing
>> connection back to connection manager.
>> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
>> "http://www.w3.org/TR/html4/strict.dtd">
>> <HTML><HEAD><TITLE>You are not authorized to view this page</TITLE>
>> <META HTTP-EQUIV="Content-Type" Content="text/html; charset=Windows-1252">
>> <STYLE type="text/css">
>> BODY { font: 8pt/12pt verdana }
>> H1 { font: 13pt/15pt verdana }
>> H2 { font: 8pt/12pt verdana }
>> A:link { color: red }
>> A:visited { color: maroon }
>> </STYLE>
>> </HEAD><BODY><TABLE width=500 border=0 cellspacing=10><TR><TD>
>>
>> <h1>You are not authorized to view this page</h1>
>> You do not have permission to view this directory or page using the
>> credentials that you supplied because your Web browser is sending a
>> WWW-Authenticate header field that the Web server is not configured to
>> accept.
>> <hr>
>> <p>Please try the following:</p>
>> <ul>
>> <li>Contact the Web site administrator if you believe you should be able
>> to view this directory or page.</li>
>> <li>Click the <a href="javascript:location.reload()">Refresh</a> button
>> to try again with different credentials.</li>
>> </ul>
>> <h2>HTTP Error 401.2 - Unauthorized: Access is denied due to server
>> configuration.<br>Internet Information Services (IIS)</h2>
>> <hr>
>> <p>Technical Information (for support personnel)</p>
>> <ul>
>> <li>Go to <a
>> href="http://go.microsoft.com/fwlink/?linkid=8180">Microsoft Product
>> Support Services</a> and perform a title search for the words
>> <b>HTTP</b> and <b>401</b>.</li>
>> <li>Open <b>IIS Help</b>, which is accessible in IIS Manager (inetmgr),
>> and search for topics titled <b>About Security</b>,
>> <b>Authentication</b>, and <b>About Custom Error Messages</b>.</li>
>> </ul>
>>
>> </TD></TR></TABLE></BODY></HTML>
>>
>>
>>
>>
>>
>> --
>>
>> Claudio Martella
>> Digital Technologies
>> Unit Research & Development - Analyst
>>
>> TIS innovation park
>> Via Siemens 19 | Siemensstr. 19
>> 39100 Bolzano | 39100 Bozen
>> Tel. +39 0471 068 123
>> Fax +39 0471 068 129
>> claudio.martella@tis.bz.it http://www.tis.bz.it
>>
>> Short information regarding use of personal data. According to Section 13
>> of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we
>> process your personal data in order to fulfil contractual and fiscal
>> obligations and also to send you information regarding our services and
>> events. Your personal data are processed with and without electronic means
>> and by respecting data subjects' rights, fundamental freedoms and dignity,
>> particularly with regard to confidentiality, personal identity and the right
>> to personal data protection. At any time and without formalities you can
>> write an e-mail to privacy@tis.bz.it in order to object the processing of
>> your personal data for the purpose of sending advertising materials and also
>> to exercise the right to access personal data and other rights referred to
>> in Section 7 of Decree 196/2003. The data controller is TIS Techno
>> Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the
>> complete information on the web site www.tis.bz.it.
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>>
--
Claudio Martella
Digital Technologies
Unit Research & Development - Analyst
TIS innovation park
Via Siemens 19 | Siemensstr. 19
39100 Bolzano | 39100 Bozen
Tel. +39 0471 068 123
Fax +39 0471 068 129
claudio.martella@tis.bz.it http://www.tis.bz.it
Short information regarding use of personal data. According to Section 13 of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we process your personal data in order to fulfil contractual and fiscal obligations and also to send you information regarding our services and events. Your personal data are processed with and without electronic means and by respecting data subjects' rights, fundamental freedoms and dignity, particularly with regard to confidentiality, personal identity and the right to personal data protection. At any time and without formalities you can write an e-mail to privacy@tis.bz.it in order to object the processing of your personal data for the purpose of sending advertising materials and also to exercise the right to access personal data and other rights referred to in Section 7 of Decree 196/2003. The data controller is TIS Techno Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the complete information on the web site www.tis.bz.it.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: httpclient 3.1 failing DIGEST authentication
Posted by Ryan Smith <ry...@gmail.com>.
You can log the browser session and then compare the working browser session
header log with the httpClient header log below and see which headers are
different. You can use tcpflow or wireshark to capture the http traffic
from your browser. Firefox also has some plugins for logging http headers
to compare with your httpClient log below. Im not too familiar with
httpClient auth, but this will show you what httpClient headers are
different from your browser's headers.
hth.
On Wed, Jan 5, 2011 at 8:08 AM, Claudio Martella <claudio.martella@tis.bz.it
> wrote:
> Hello list,
>
> I'm using Apache Nutch to crawl my intranet which is under DIGEST
> authentication (nutch is using httpclient 3.1).
> As the client is failing the auth with the same credentials my browser
> is succeeding, I wrote an example app to try figure out what's going wrong.
>
> Here it is:
>
> HttpClient client = new HttpClient();
> client.getParams().setAuthenticationPreemptive(true);
> Credentials defaultcreds = new
> UsernamePasswordCredentials("user", "*******");
> List authPrefs = new ArrayList();
> authPrefs.add(AuthPolicy.DIGEST);
> client.getParams().setParameter(AuthPolicy.AUTH_SCHEME_PRIORITY,
> authPrefs);
> client.getState().setCredentials(AuthScope.ANY, defaultcreds);
> HttpMethod method = new GetMethod("http://192.168.10.209:8090");
>
>
> What I can see from the logs is that the client is trying to
> authenticate with Basic authentication but the server expects NTLM and
> only NTLM. Am I reading it correctly?
> Why isn't it trying to authenticate with Digest as requested?
>
>
> Here are the logs:
>
> 2011/01/05 13:25:07:566 CET [DEBUG] HttpClient - Java version: 1.6.0_22
> 2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Java vendor: Apple Inc.
> 2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Java class path:
>
> /Users/hammer/TIS/java-hacking/auth-test/target/classes:/Users/hammer/.m2/repository/commons-codec/commons-codec/1.2/commons-codec-1.2.jar:/Users/hammer/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/Users/hammer/.m2/repository/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.jar:/Users/hammer/.m2/repository/org/apache/httpcomponents/httpclient/4.0.3/httpclient-4.0.3.jar:/Users/hammer/.m2/repository/org/apache/httpcomponents/httpcore/4.0.1/httpcore-4.0.1.jar:/Users/hammer/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar:/Users/hammer/.m2/repository/log4j/log4j/1.2.14/log4j-1.2.14.jar
> 2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Operating system name:
> Mac OS X
> 2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Operating system
> architecture: x86_64
> 2011/01/05 13:25:07:574 CET [DEBUG] HttpClient - Operating system
> version: 10.5.8
> 2011/01/05 13:25:07:697 CET [DEBUG] HttpClient - SUN 1.6: SUN (DSA
> key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom;
> X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
> CertPathBuilder; LDAP, Collection CertStores, JavaPolicy Policy;
> JavaLoginConfig Configuration)
> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - Apple 1.0: Apple
> Provider (implements DES, Triple DES, AES, Blowfish, PBE,
> Diffie-Hellman, HMAC/MD5, HMAC/SHA1)
> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunRsaSign 1.5: Sun RSA
> signature provider
> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunJSSE 1.6: Sun JSSE
> provider(PKCS12, SunX509 key/trust factories, SSLv3, TLSv1)
> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunJCE 1.6: SunJCE
> Provider (implements RSA, DES, Triple DES, AES, Blowfish, ARCFOUR, RC2,
> PBE, Diffie-Hellman, HMAC)
> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunJGSS 1.0: Sun
> (Kerberos v5, SPNEGO)
> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunSASL 1.5: Sun SASL
> provider(implements client mechanisms for: DIGEST-MD5, GSSAPI, EXTERNAL,
> PLAIN, CRAM-MD5; server mechanisms for: DIGEST-MD5, GSSAPI, CRAM-MD5)
> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - XMLDSig 1.0: XMLDSig
> (DOM XMLSignatureFactory; DOM KeyInfoFactory)
> 2011/01/05 13:25:07:698 CET [DEBUG] HttpClient - SunPCSC 1.6: Sun PC/SC
> provider
> 2011/01/05 13:25:07:703 CET [DEBUG] DefaultHttpParams - Set parameter
> http.useragent = Jakarta Commons-HttpClient/3.1
> 2011/01/05 13:25:07:705 CET [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.version = HTTP/1.1
> 2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
> http.connection-manager.class = class
> org.apache.commons.httpclient.SimpleHttpConnectionManager
> 2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.cookie-policy = default
> 2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.element-charset = US-ASCII
> 2011/01/05 13:25:07:706 CET [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.content-charset = ISO-8859-1
> 2011/01/05 13:25:07:708 CET [DEBUG] DefaultHttpParams - Set parameter
> http.method.retry-handler =
> org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@41fc2fb
> 2011/01/05 13:25:07:708 CET [DEBUG] DefaultHttpParams - Set parameter
> http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE,
> dd-MMM-yy HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy
> HH:mm:ss z, EEE, dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE
> dd-MMM-yyyy HH:mm:ss z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy
> HH-mm-ss z, EEE dd-MMM-yy HH:mm:ss z, EEE dd MMM yy HH:mm:ss z,
> EEE,dd-MMM-yy HH:mm:ss z, EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy
> HH:mm:ss z]
> 2011/01/05 13:25:07:713 CET [DEBUG] DefaultHttpParams - Set parameter
> http.authentication.preemptive = true
> 2011/01/05 13:25:07:715 CET [DEBUG] DefaultHttpParams - Set parameter
> http.auth.scheme-priority = [Digest]
> 2011/01/05 13:25:07:764 CET [DEBUG] DefaultHttpParams - Set parameter
> http.method.retry-handler =
> org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@450e790c
> 2011/01/05 13:25:07:774 CET [DEBUG] HttpMethodDirector - Preemptively
> sending default basic credentials
> 2011/01/05 13:25:07:783 CET [DEBUG] HttpMethodDirector - Authenticating
> with BASIC <any realm>@192.168.10.210:8090
> 2011/01/05 13:25:07:783 CET [DEBUG] HttpMethodParams - Credential
> charset not configured, using HTTP element charset
> 2011/01/05 13:25:07:785 CET [DEBUG] HttpConnection - Open connection to
> 192.168.10.210:8090
> 2011/01/05 13:25:07:818 CET [DEBUG] header - >> "GET / HTTP/1.1[\r][\n]"
> 2011/01/05 13:25:07:819 CET [DEBUG] HttpMethodBase - Adding Host request
> header
> 2011/01/05 13:25:07:832 CET [DEBUG] header - >> "Authorization: Basic
> ****************************[\r][\n]"
> 2011/01/05 13:25:07:832 CET [DEBUG] header - >> "User-Agent: Jakarta
> Commons-HttpClient/3.1[\r][\n]"
> 2011/01/05 13:25:07:833 CET [DEBUG] header - >> "Host:
> 192.168.10.210:8090[\r][\n]"
> 2011/01/05 13:25:07:833 CET [DEBUG] header - >> "[\r][\n]"
> 2011/01/05 13:25:07:835 CET [DEBUG] header - << "HTTP/1.1 401
> Unauthorized[\r][\n]"
> 2011/01/05 13:25:07:835 CET [DEBUG] header - << "HTTP/1.1 401
> Unauthorized[\r][\n]"
> 2011/01/05 13:25:07:836 CET [DEBUG] header - << "Content-Length:
> 1656[\r][\n]"
> 2011/01/05 13:25:07:836 CET [DEBUG] header - << "Content-Type:
> text/html[\r][\n]"
> 2011/01/05 13:25:07:836 CET [DEBUG] header - << "Server:
> Microsoft-IIS/6.0[\r][\n]"
> 2011/01/05 13:25:07:837 CET [DEBUG] header - << "WWW-Authenticate:
> Negotiate[\r][\n]"
> 2011/01/05 13:25:07:837 CET [DEBUG] header - << "WWW-Authenticate:
> NTLM[\r][\n]"
> 2011/01/05 13:25:07:837 CET [DEBUG] header - << "X-Powered-By:
> ASP.NET[\r][\n]"
> 2011/01/05 13:25:07:837 CET [DEBUG] header - << "Date: Wed, 05 Jan 2011
> 12:25:07 GMT[\r][\n]"
> 2011/01/05 13:25:07:837 CET [DEBUG] header - << "[\r][\n]"
> 2011/01/05 13:25:07:838 CET [DEBUG] HttpMethodDirector - Authorization
> required
> 2011/01/05 13:25:07:839 CET [DEBUG] AuthChallengeProcessor - Supported
> authentication schemes in the order of preference: [Digest]
> 2011/01/05 13:25:07:839 CET [DEBUG] AuthChallengeProcessor - Challenge
> for Digest authentication scheme not available
> 2011/01/05 13:25:07:840 CET [WARN] HttpMethodDirector - Unable to
> respond to any of these challenges: {ntlm=NTLM, negotiate=Negotiate}
> Method failed: HTTP/1.1 401 Unauthorized
> 2011/01/05 13:25:07:840 CET [DEBUG] HttpMethodBase - Buffering response
> body
> 2011/01/05 13:25:07:841 CET [DEBUG] HttpMethodBase - Resorting to
> protocol version default close connection policy
> 2011/01/05 13:25:07:841 CET [DEBUG] HttpMethodBase - Should NOT close
> connection, using HTTP/1.1
> 2011/01/05 13:25:07:841 CET [DEBUG] HttpConnection - Releasing
> connection back to connection manager.
> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
> "http://www.w3.org/TR/html4/strict.dtd">
> <HTML><HEAD><TITLE>You are not authorized to view this page</TITLE>
> <META HTTP-EQUIV="Content-Type" Content="text/html; charset=Windows-1252">
> <STYLE type="text/css">
> BODY { font: 8pt/12pt verdana }
> H1 { font: 13pt/15pt verdana }
> H2 { font: 8pt/12pt verdana }
> A:link { color: red }
> A:visited { color: maroon }
> </STYLE>
> </HEAD><BODY><TABLE width=500 border=0 cellspacing=10><TR><TD>
>
> <h1>You are not authorized to view this page</h1>
> You do not have permission to view this directory or page using the
> credentials that you supplied because your Web browser is sending a
> WWW-Authenticate header field that the Web server is not configured to
> accept.
> <hr>
> <p>Please try the following:</p>
> <ul>
> <li>Contact the Web site administrator if you believe you should be able
> to view this directory or page.</li>
> <li>Click the <a href="javascript:location.reload()">Refresh</a> button
> to try again with different credentials.</li>
> </ul>
> <h2>HTTP Error 401.2 - Unauthorized: Access is denied due to server
> configuration.<br>Internet Information Services (IIS)</h2>
> <hr>
> <p>Technical Information (for support personnel)</p>
> <ul>
> <li>Go to <a
> href="http://go.microsoft.com/fwlink/?linkid=8180">Microsoft Product
> Support Services</a> and perform a title search for the words
> <b>HTTP</b> and <b>401</b>.</li>
> <li>Open <b>IIS Help</b>, which is accessible in IIS Manager (inetmgr),
> and search for topics titled <b>About Security</b>,
> <b>Authentication</b>, and <b>About Custom Error Messages</b>.</li>
> </ul>
>
> </TD></TR></TABLE></BODY></HTML>
>
>
>
>
>
> --
>
> Claudio Martella
> Digital Technologies
> Unit Research & Development - Analyst
>
> TIS innovation park
> Via Siemens 19 | Siemensstr. 19
> 39100 Bolzano | 39100 Bozen
> Tel. +39 0471 068 123
> Fax +39 0471 068 129
> claudio.martella@tis.bz.it http://www.tis.bz.it
>
> Short information regarding use of personal data. According to Section 13
> of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we
> process your personal data in order to fulfil contractual and fiscal
> obligations and also to send you information regarding our services and
> events. Your personal data are processed with and without electronic means
> and by respecting data subjects' rights, fundamental freedoms and dignity,
> particularly with regard to confidentiality, personal identity and the right
> to personal data protection. At any time and without formalities you can
> write an e-mail to privacy@tis.bz.it in order to object the processing of
> your personal data for the purpose of sending advertising materials and also
> to exercise the right to access personal data and other rights referred to
> in Section 7 of Decree 196/2003. The data controller is TIS Techno
> Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the
> complete information on the web site www.tis.bz.it.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>