You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Sebastiaan van Erk <se...@sebster.com> on 2008/07/31 18:06:51 UTC
Preemptive auth and mixed auth types and alpha5
Hi,
I have a question a preemptive authentication and auth types in alhpa5.
First of all, I'm doing some non-repeatable requests, but I always do 1
repeatable short request first.
I can use this first request to authenticate to the proxy (figuring out
the out type etc), but the following requests should be preemptive if
possible. I use all three possible auth types (basic, digest, and NTLM).
First question: is the above doable with httpclient 4.0-alpha5?
Second question: when I register credentials with httpclient, I can only
set them once. So how can set the correct credentials if I don't yet
know the auth type? Or should I use the first request to determine the
auth type? Currently there are UsernamePasswordCredentials (for BASIC
and DIGEST) and NTCredentials (for NTLM). Or is it possible to *always*
just use NTCredentials for the other auth types as well?
Third question: in alpha5 the following code no longer compiles:
HttpClientParams.setAuthenticationPreemptive(true);
What is the correct way to use preemptive authentication now?
Regards,
Sebastiaan
Re: Preemptive auth and mixed auth types and alpha5
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2008-08-20 at 11:44 +0200, Sebastiaan van Erk wrote:
> Hi,
>
> Unfortunately the HEAD option is not very stable either. The nonce is
> usable only once as well, so in the next post it is no good anymore.
> However, if I set the nonce preemptively using the request/response
> interceptor method ISA will say 407 but httpclient will not try again,
> which will mean that due to the response interceptor the new DIGEST auth
> has been saved and it will work for the next POST.
>
> I don't really understand why httpclient does retry the request when
> there is no auth preemptively set, but does not retry the request when
> there is.
>
> Also I'm afraid this solution is very brittle: if the HTTP target
> requires any kind of digest authentication I'm sure this will completely
> fail.
>
I am not sure I understand the problem. Could you please post wire logs
of both sessions?
> The only thing I can really think of is to somehow make the POST
> repeatable (buffer it?). Is the value of isRepeatable() allowed to
> change?
No, but you can decorate the entity with a buffering wrapper similar to
the BufferedHttpEntity.
Hope this helps
Oleg
> I.e. it is repeatable as long as we're in the first 4096 bytes
> of the stream (the buffer), but after that, it is no more? I can't
> buffer multimegabytes, but as long as I can buffer the enough to handle
> the authentication, it should do the trick...
>
> Regards,
> Sebastiaan
>
> Sebastiaan van Erk wrote:
> > Just as I replied to this message, I noticed that I had not tried the
> > HEAD option which you suggested.
> >
> > I tried it, and it does in fact work with DIGEST.
> >
> > Thanks!
> >
> > Regards,
> > Sebastiaan
> >
> > Sebastiaan van Erk wrote:
> >> Hi,
> >>
> >> I'm still trying to get different types of authentication to work,
> >> this time I'm testing with Microsoft ISA Server 2006 (which seems
> >> pretty broken). I got basic to work with a ResponseInterceptor to pick
> >> of the auth from a successful small request preceding a large request,
> >> and then using preemptive from then on the large POSTS. However, I'm
> >> having trouble with DIGEST.
> >>
> >>>> Oleg Kalnichevski wrote:
> >>>>
> >>>>> HttpClient 4.0 can be customized to support preemptive
> >>>>> authentication using BASIC or DIGEST schemes. NTLM cannot be used
> >>>>> preemptively in principle.
> >>
> >> Just to clarify my understanding: DIGEST can only be used preemptively
> >> when the server accepts the reuse of a previously used nonce right?
> >> That is, if the proxy server requires a new DIGEST challenge/response
> >> every request, then preemptive DIGEST auth will (by definition) fail?
> >>
> >>>> Ok, I was afraid of that. Does that mean that I am forced to use
> >>>> expect/continue with non-repeatable requests?
> >>>>
> >>> I am afraid so. Another alternative would be to execute a GET or a HEAD
> >>> request to make sure credentials are OK before executing a POST with a
> >>> large entity. The good thing about NTLM authentication scheme is that
> >>> one has only to authenticate once. NTLM authentication is connection
> >>> based. A persistent HTTP connection will retain its NTLM context as long
> >>> as it remains open.
> >>
> >> Ok, I'm testing with MS ISA 2006 as mentioned above. It seems to be
> >> very broken: when doing expect/continue it will *ALWAYS* respond 100
> >> Continue when doing a POST, only to fail with a 407 the second you
> >> start sending data. This breaks non-repeatable POSTS with DIGEST
> >> authentication, and I can't use the preemptive DIGEST using
> >> authentication from a previous request, since ISA requires a new
> >> challenge/response on the every request (even in the same connection).
> >>
> >>>>> The use of preemptive authentication is discouraged (or at least
> >>>>> not promoted). However, one can easily add preemptive
> >>>>> authentication capabilities using custom protocol interceptors. See
> >>>>> samples above.
> >>>>
> >>>> Ok, I'll give the interceptors a shot. The reason I want (need?)
> >>>> preemptive authentication is because some proxies do not support
> >>>> expect/continue and I have non-repeatable posts (multi-megabyte size).
> >>>>
> >>> An HTTP GET or HEAD preceding a POST with a large content entity is the
> >>> way to go.
> >>
> >> This works with NTLM (connection based), but not with DIGEST (at least
> >> on ISA). Do you have any ideas how to go about it with DIGEST?
> >>
> >> Regards,
> >> Sebastiaan
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Preemptive auth and mixed auth types and alpha5
Posted by Sebastiaan van Erk <se...@sebster.com>.
Hi,
Unfortunately the HEAD option is not very stable either. The nonce is
usable only once as well, so in the next post it is no good anymore.
However, if I set the nonce preemptively using the request/response
interceptor method ISA will say 407 but httpclient will not try again,
which will mean that due to the response interceptor the new DIGEST auth
has been saved and it will work for the next POST.
I don't really understand why httpclient does retry the request when
there is no auth preemptively set, but does not retry the request when
there is.
Also I'm afraid this solution is very brittle: if the HTTP target
requires any kind of digest authentication I'm sure this will completely
fail.
The only thing I can really think of is to somehow make the POST
repeatable (buffer it?). Is the value of isRepeatable() allowed to
change? I.e. it is repeatable as long as we're in the first 4096 bytes
of the stream (the buffer), but after that, it is no more? I can't
buffer multimegabytes, but as long as I can buffer the enough to handle
the authentication, it should do the trick...
Regards,
Sebastiaan
Sebastiaan van Erk wrote:
> Just as I replied to this message, I noticed that I had not tried the
> HEAD option which you suggested.
>
> I tried it, and it does in fact work with DIGEST.
>
> Thanks!
>
> Regards,
> Sebastiaan
>
> Sebastiaan van Erk wrote:
>> Hi,
>>
>> I'm still trying to get different types of authentication to work,
>> this time I'm testing with Microsoft ISA Server 2006 (which seems
>> pretty broken). I got basic to work with a ResponseInterceptor to pick
>> of the auth from a successful small request preceding a large request,
>> and then using preemptive from then on the large POSTS. However, I'm
>> having trouble with DIGEST.
>>
>>>> Oleg Kalnichevski wrote:
>>>>
>>>>> HttpClient 4.0 can be customized to support preemptive
>>>>> authentication using BASIC or DIGEST schemes. NTLM cannot be used
>>>>> preemptively in principle.
>>
>> Just to clarify my understanding: DIGEST can only be used preemptively
>> when the server accepts the reuse of a previously used nonce right?
>> That is, if the proxy server requires a new DIGEST challenge/response
>> every request, then preemptive DIGEST auth will (by definition) fail?
>>
>>>> Ok, I was afraid of that. Does that mean that I am forced to use
>>>> expect/continue with non-repeatable requests?
>>>>
>>> I am afraid so. Another alternative would be to execute a GET or a HEAD
>>> request to make sure credentials are OK before executing a POST with a
>>> large entity. The good thing about NTLM authentication scheme is that
>>> one has only to authenticate once. NTLM authentication is connection
>>> based. A persistent HTTP connection will retain its NTLM context as long
>>> as it remains open.
>>
>> Ok, I'm testing with MS ISA 2006 as mentioned above. It seems to be
>> very broken: when doing expect/continue it will *ALWAYS* respond 100
>> Continue when doing a POST, only to fail with a 407 the second you
>> start sending data. This breaks non-repeatable POSTS with DIGEST
>> authentication, and I can't use the preemptive DIGEST using
>> authentication from a previous request, since ISA requires a new
>> challenge/response on the every request (even in the same connection).
>>
>>>>> The use of preemptive authentication is discouraged (or at least
>>>>> not promoted). However, one can easily add preemptive
>>>>> authentication capabilities using custom protocol interceptors. See
>>>>> samples above.
>>>>
>>>> Ok, I'll give the interceptors a shot. The reason I want (need?)
>>>> preemptive authentication is because some proxies do not support
>>>> expect/continue and I have non-repeatable posts (multi-megabyte size).
>>>>
>>> An HTTP GET or HEAD preceding a POST with a large content entity is the
>>> way to go.
>>
>> This works with NTLM (connection based), but not with DIGEST (at least
>> on ISA). Do you have any ideas how to go about it with DIGEST?
>>
>> Regards,
>> Sebastiaan
Re: Preemptive auth and mixed auth types and alpha5
Posted by Sebastiaan van Erk <se...@sebster.com>.
Just as I replied to this message, I noticed that I had not tried the
HEAD option which you suggested.
I tried it, and it does in fact work with DIGEST.
Thanks!
Regards,
Sebastiaan
Sebastiaan van Erk wrote:
> Hi,
>
> I'm still trying to get different types of authentication to work, this
> time I'm testing with Microsoft ISA Server 2006 (which seems pretty
> broken). I got basic to work with a ResponseInterceptor to pick of the
> auth from a successful small request preceding a large request, and then
> using preemptive from then on the large POSTS. However, I'm having
> trouble with DIGEST.
>
>>> Oleg Kalnichevski wrote:
>>>
>>>> HttpClient 4.0 can be customized to support preemptive
>>>> authentication using BASIC or DIGEST schemes. NTLM cannot be used
>>>> preemptively in principle.
>
> Just to clarify my understanding: DIGEST can only be used preemptively
> when the server accepts the reuse of a previously used nonce right? That
> is, if the proxy server requires a new DIGEST challenge/response every
> request, then preemptive DIGEST auth will (by definition) fail?
>
>>> Ok, I was afraid of that. Does that mean that I am forced to use
>>> expect/continue with non-repeatable requests?
>>>
>> I am afraid so. Another alternative would be to execute a GET or a HEAD
>> request to make sure credentials are OK before executing a POST with a
>> large entity. The good thing about NTLM authentication scheme is that
>> one has only to authenticate once. NTLM authentication is connection
>> based. A persistent HTTP connection will retain its NTLM context as long
>> as it remains open.
>
> Ok, I'm testing with MS ISA 2006 as mentioned above. It seems to be very
> broken: when doing expect/continue it will *ALWAYS* respond 100 Continue
> when doing a POST, only to fail with a 407 the second you start sending
> data. This breaks non-repeatable POSTS with DIGEST authentication, and I
> can't use the preemptive DIGEST using authentication from a previous
> request, since ISA requires a new challenge/response on the every
> request (even in the same connection).
>
>>>> The use of preemptive authentication is discouraged (or at least not
>>>> promoted). However, one can easily add preemptive authentication
>>>> capabilities using custom protocol interceptors. See samples above.
>>>
>>> Ok, I'll give the interceptors a shot. The reason I want (need?)
>>> preemptive authentication is because some proxies do not support
>>> expect/continue and I have non-repeatable posts (multi-megabyte size).
>>>
>> An HTTP GET or HEAD preceding a POST with a large content entity is the
>> way to go.
>
> This works with NTLM (connection based), but not with DIGEST (at least
> on ISA). Do you have any ideas how to go about it with DIGEST?
>
> Regards,
> Sebastiaan
Re: Preemptive auth and mixed auth types and alpha5
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2008-08-20 at 11:19 +0200, Sebastiaan van Erk wrote:
> Hi,
>
> I'm still trying to get different types of authentication to work, this
> time I'm testing with Microsoft ISA Server 2006 (which seems pretty
> broken). I got basic to work with a ResponseInterceptor to pick of the
> auth from a successful small request preceding a large request, and then
> using preemptive from then on the large POSTS. However, I'm having
> trouble with DIGEST.
>
> >> Oleg Kalnichevski wrote:
> >>
> >>> HttpClient 4.0 can be customized to support preemptive authentication
> >>> using BASIC or DIGEST schemes. NTLM cannot be used preemptively in
> >>> principle.
>
> Just to clarify my understanding: DIGEST can only be used preemptively
> when the server accepts the reuse of a previously used nonce right?
Absolutely right.
> That
> is, if the proxy server requires a new DIGEST challenge/response every
> request, then preemptive DIGEST auth will (by definition) fail?
>
Yes, it will
> >> Ok, I was afraid of that. Does that mean that I am forced to use
> >> expect/continue with non-repeatable requests?
> >>
> > I am afraid so. Another alternative would be to execute a GET or a HEAD
> > request to make sure credentials are OK before executing a POST with a
> > large entity. The good thing about NTLM authentication scheme is that
> > one has only to authenticate once. NTLM authentication is connection
> > based. A persistent HTTP connection will retain its NTLM context as long
> > as it remains open.
>
> Ok, I'm testing with MS ISA 2006 as mentioned above. It seems to be very
> broken: when doing expect/continue it will *ALWAYS* respond 100 Continue
> when doing a POST, only to fail with a 407 the second you start sending
> data.
I am not aware of any HTTP proxy that supports the expect/continue
handshaking fully. I believe Squid also supports it only partially.
> This breaks non-repeatable POSTS with DIGEST authentication, and I
> can't use the preemptive DIGEST using authentication from a previous
> request, since ISA requires a new challenge/response on the every
> request (even in the same connection).
>
> >>> The use of preemptive authentication is discouraged (or at least not
> >>> promoted). However, one can easily add preemptive authentication
> >>> capabilities using custom protocol interceptors. See samples above.
> >>
> >> Ok, I'll give the interceptors a shot. The reason I want (need?)
> >> preemptive authentication is because some proxies do not support
> >> expect/continue and I have non-repeatable posts (multi-megabyte size).
> >>
> > An HTTP GET or HEAD preceding a POST with a large content entity is the
> > way to go.
>
> This works with NTLM (connection based), but not with DIGEST (at least
> on ISA). Do you have any ideas how to go about it with DIGEST?
>
I do not see an easy way around this problem. The only possibility would
be to configure the ISA to update the nonce value less frequently.
Oleg
> Regards,
> Sebastiaan
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Preemptive auth and mixed auth types and alpha5
Posted by Sebastiaan van Erk <se...@sebster.com>.
Hi,
I'm still trying to get different types of authentication to work, this
time I'm testing with Microsoft ISA Server 2006 (which seems pretty
broken). I got basic to work with a ResponseInterceptor to pick of the
auth from a successful small request preceding a large request, and then
using preemptive from then on the large POSTS. However, I'm having
trouble with DIGEST.
>> Oleg Kalnichevski wrote:
>>
>>> HttpClient 4.0 can be customized to support preemptive authentication
>>> using BASIC or DIGEST schemes. NTLM cannot be used preemptively in
>>> principle.
Just to clarify my understanding: DIGEST can only be used preemptively
when the server accepts the reuse of a previously used nonce right? That
is, if the proxy server requires a new DIGEST challenge/response every
request, then preemptive DIGEST auth will (by definition) fail?
>> Ok, I was afraid of that. Does that mean that I am forced to use
>> expect/continue with non-repeatable requests?
>>
> I am afraid so. Another alternative would be to execute a GET or a HEAD
> request to make sure credentials are OK before executing a POST with a
> large entity. The good thing about NTLM authentication scheme is that
> one has only to authenticate once. NTLM authentication is connection
> based. A persistent HTTP connection will retain its NTLM context as long
> as it remains open.
Ok, I'm testing with MS ISA 2006 as mentioned above. It seems to be very
broken: when doing expect/continue it will *ALWAYS* respond 100 Continue
when doing a POST, only to fail with a 407 the second you start sending
data. This breaks non-repeatable POSTS with DIGEST authentication, and I
can't use the preemptive DIGEST using authentication from a previous
request, since ISA requires a new challenge/response on the every
request (even in the same connection).
>>> The use of preemptive authentication is discouraged (or at least not
>>> promoted). However, one can easily add preemptive authentication
>>> capabilities using custom protocol interceptors. See samples above.
>>
>> Ok, I'll give the interceptors a shot. The reason I want (need?)
>> preemptive authentication is because some proxies do not support
>> expect/continue and I have non-repeatable posts (multi-megabyte size).
>>
> An HTTP GET or HEAD preceding a POST with a large content entity is the
> way to go.
This works with NTLM (connection based), but not with DIGEST (at least
on ISA). Do you have any ideas how to go about it with DIGEST?
Regards,
Sebastiaan
Re: Preemptive auth and mixed auth types and alpha5
Posted by Oleg Kalnichevski <ol...@apache.org>.
Sebastiaan van Erk wrote:
> Hi,
>
> Thanks for your helpful answers, as usual. :-)
>
> Oleg Kalnichevski wrote:
>
>> HttpClient 4.0 can be customized to support preemptive authentication
>> using BASIC or DIGEST schemes. NTLM cannot be used preemptively in
>> principle.
>
Hi Sebastiaan
> Ok, I was afraid of that. Does that mean that I am forced to use
> expect/continue with non-repeatable requests?
>
I am afraid so. Another alternative would be to execute a GET or a HEAD
request to make sure credentials are OK before executing a POST with a
large entity. The good thing about NTLM authentication scheme is that
one has only to authenticate once. NTLM authentication is connection
based. A persistent HTTP connection will retain its NTLM context as long
as it remains open.
>>> Second question: when I register credentials with httpclient, I can
>>> only set them once. So how can set the correct credentials if I
>>> don't yet know the auth type? Or should I use the first request to
>>> determine the auth type? Currently there are
>>> UsernamePasswordCredentials (for BASIC and DIGEST) and NTCredentials
>>> (for NTLM). Or is it possible to *always* just use NTCredentials for
>>> the other auth types as well?
>
>> Yes, it is. NTCredentials are perfectly valid for non-NTLM
>> authentication schemes.
>
> Ok, that's great. :-) Does it just ignore the domain and workstation
> parts then?
>
HttpClient will ignore the workstation part but will use the domain part
to construct a fully qualified domain user name.
> Currently I'm using the
> InetAddress.getLocalHost().getCanonicalHostName() to initialize the
> workstation/hostname, and I'm splitting the supplied into a domain and
> username (so if the user types DOMAIN\username I split it into the two
> components). The only thing this might then break is if BASIC or
> DIGEST is used and it was meant to send the whole DOMAIN\username
> combination as the user name (I have seen it used that way in real
> life configurations, i.e., you have to authenticate with BASIC using
> DOMAIN\username as the user name).
>
If you supply user credentials as an NTCredentials instance, HttpClient
will automatically use a fully qualified domain user name with BASIC and
DIGEST schemes.
>> The use of preemptive authentication is discouraged (or at least not
>> promoted). However, one can easily add preemptive authentication
>> capabilities using custom protocol interceptors. See samples above.
>
> Ok, I'll give the interceptors a shot. The reason I want (need?)
> preemptive authentication is because some proxies do not support
> expect/continue and I have non-repeatable posts (multi-megabyte size).
>
An HTTP GET or HEAD preceding a POST with a large content entity is the
way to go.
Hope this helps
Oleg
>> Hope this helps
>
> Yes, very much. :-)
>
>> Oleg
>
> Regards,
> Sebastiaan
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Preemptive auth and mixed auth types and alpha5
Posted by Sebastiaan van Erk <se...@sebster.com>.
Hi,
Thanks for your helpful answers, as usual. :-)
Oleg Kalnichevski wrote:
> HttpClient 4.0 can be customized to support preemptive authentication
> using BASIC or DIGEST schemes. NTLM cannot be used preemptively in
> principle.
Ok, I was afraid of that. Does that mean that I am forced to use
expect/continue with non-repeatable requests?
>> Second question: when I register credentials with httpclient, I can
>> only set them once. So how can set the correct credentials if I don't
>> yet know the auth type? Or should I use the first request to determine
>> the auth type? Currently there are UsernamePasswordCredentials (for
>> BASIC and DIGEST) and NTCredentials (for NTLM). Or is it possible to
>> *always* just use NTCredentials for the other auth types as well?
> Yes, it is. NTCredentials are perfectly valid for non-NTLM
> authentication schemes.
Ok, that's great. :-) Does it just ignore the domain and workstation
parts then?
Currently I'm using the
InetAddress.getLocalHost().getCanonicalHostName() to initialize the
workstation/hostname, and I'm splitting the supplied into a domain and
username (so if the user types DOMAIN\username I split it into the two
components). The only thing this might then break is if BASIC or DIGEST
is used and it was meant to send the whole DOMAIN\username combination
as the user name (I have seen it used that way in real life
configurations, i.e., you have to authenticate with BASIC using
DOMAIN\username as the user name).
> The use of preemptive authentication is discouraged (or at least not
> promoted). However, one can easily add preemptive authentication
> capabilities using custom protocol interceptors. See samples above.
Ok, I'll give the interceptors a shot. The reason I want (need?)
preemptive authentication is because some proxies do not support
expect/continue and I have non-repeatable posts (multi-megabyte size).
> Hope this helps
Yes, very much. :-)
> Oleg
Regards,
Sebastiaan
Re: Preemptive auth and mixed auth types and alpha5
Posted by Oleg Kalnichevski <ol...@apache.org>.
Sebastiaan van Erk wrote:
> Hi,
>
> I have a question a preemptive authentication and auth types in alhpa5.
>
> First of all, I'm doing some non-repeatable requests, but I always do
> 1 repeatable short request first.
>
> I can use this first request to authenticate to the proxy (figuring
> out the out type etc), but the following requests should be preemptive
> if possible. I use all three possible auth types (basic, digest, and
> NTLM).
>
> First question: is the above doable with httpclient 4.0-alpha5?
>
HttpClient 4.0 can be customized to support preemptive authentication
using BASIC or DIGEST schemes. NTLM cannot be used preemptively in
principle.
http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/module-client/src/examples/org/apache/http/examples/client/ClientPreemptiveBasicAuthentication.java
http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/module-client/src/examples/org/apache/http/examples/client/ClientPreemptiveDigestAuthentication.java
> Second question: when I register credentials with httpclient, I can
> only set them once. So how can set the correct credentials if I don't
> yet know the auth type? Or should I use the first request to determine
> the auth type? Currently there are UsernamePasswordCredentials (for
> BASIC and DIGEST) and NTCredentials (for NTLM). Or is it possible to
> *always* just use NTCredentials for the other auth types as well?
Yes, it is. NTCredentials are perfectly valid for non-NTLM
authentication schemes.
>
> Third question: in alpha5 the following code no longer compiles:
>
> HttpClientParams.setAuthenticationPreemptive(true);
>
> What is the correct way to use preemptive authentication now?
>
The use of preemptive authentication is discouraged (or at least not
promoted). However, one can easily add preemptive authentication
capabilities using custom protocol interceptors. See samples above.
Hope this helps
Oleg
> Regards,
> Sebastiaan
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org