You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Ron Jacobs <Ro...@Reardencommerce.com> on 2011/04/21 19:12:49 UTC
Full NTLMv2 Support Achieved Easily (Was: NTLM authentication with
a UPN instead of domain and user name)
(I apologize for the delay in posting this update. It was delayed due to
matters outside of my control. Nonetheless, I am sure that anyone
struggling to gain NTLMv2 support with version 4.1.x will be helped by
taking a look at the following:)
------------------------------------------------------------------------
I have tried JCIFS and was able to get all of my test cases to pass.
But, it doesn't support NTLMv2 "out of the box" because its
NegotiateFlags (see section 2.2.2.5 of the Microsoft's [MS-NLMP] spec)
is insufficient.
My implementation is based upon the example that used to be found at:
http://hc.apache.org/httpcomponents-client/ntlm.html
I was able to locate the info on the way-back machine at:
http://replay.waybackmachine.org/20090611051135/http://hc.apache.org/httpcomponents-client/ntlm.html
Since the built-in NTLM implementation is insufficient, the maintainers
may wish to restore this page, with the slight adjustment that I found
necessary, or with the following code, which I have verified works with
both NTLMv1 and NTLMv2 authentication:
------------------------------------------------------------------------
import java.io.IOException;
import jcifs.ntlmssp.NtlmFlags;
import jcifs.ntlmssp.Type1Message;
import jcifs.ntlmssp.Type2Message;
import jcifs.ntlmssp.Type3Message;
import jcifs.util.Base64;
import org.apache.http.auth.AuthScheme;
import org.apache.http.auth.AuthSchemeFactory;
import org.apache.http.client.params.AuthPolicy;
import org.apache.http.impl.auth.NTLMEngine;
import org.apache.http.impl.auth.NTLMEngineException;
import org.apache.http.impl.auth.NTLMScheme;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.HttpParams;
public final class JCIFSEngine implements NTLMEngine {
private static final int TYPE_1_FLAGS =
NtlmFlags.NTLMSSP_NEGOTIATE_56 |
NtlmFlags.NTLMSSP_NEGOTIATE_128 |
NtlmFlags.NTLMSSP_NEGOTIATE_NTLM2 |
NtlmFlags.NTLMSSP_NEGOTIATE_ALWAYS_SIGN |
NtlmFlags.NTLMSSP_REQUEST_TARGET;
public String generateType1Msg(final String domain, final String workstation) throws NTLMEngineException {
final Type1Message type1Message = new Type1Message(TYPE_1_FLAGS, domain, workstation);
return Base64.encode(type1Message.toByteArray());
}
public String generateType3Msg(final String username, final String password, final String domain,
final String workstation, final String challenge) throws NTLMEngineException {
Type2Message type2Message;
try {
type2Message = new Type2Message(Base64.decode(challenge));
}
catch (final IOException exception) {
throw new NTLMEngineException("Invalid NTLM type 2 message", exception);
}
final int type2Flags = type2Message.getFlags();
final int type3Flags = type2Flags & (0xffffffff ^ (
NtlmFlags.NTLMSSP_TARGET_TYPE_DOMAIN |
NtlmFlags.NTLMSSP_TARGET_TYPE_SERVER));
final Type3Message type3Message = new Type3Message(type2Message, password, domain, username, workstation, type3Flags);
return Base64.encode(type3Message.toByteArray());
}
public static void register(final DefaultHttpClient httpClient) {
httpClient.getAuthSchemes().register(AuthPolicy.NTLM, new AuthSchemeFactory() {
public AuthScheme newInstance(final HttpParams httpParams) {
return new NTLMScheme(new JCIFSEngine());
}
});
}
}
------------------------------------------------------------------------
Then, just insert a call to JCIFSEngine.register() for each newly
created DefaultHttpClient instance.
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Friday, March 11, 2011 10:07 AM
To: HttpClient User Discussion
Subject: Re: NTLM authentication with a UPN instead of domain and user name
On Fri, 2011-03-11 at 17:47 +0000, Ron Jacobs wrote:
> I am using NTLM authentication and I find that when porting from 3.0.2
> to 4.1, I have several questions:
>
> 1) Authenticating with domain and user works but with a UPN (like
> "user@domain.local") authentication fails although both authenticate
> fine with my previous 3.0.2 code. Why does the UPN-style not work?
>
HttpClient 4.1 has a completely new NTML engine contributed by an
external project. Apparently it does not work with UPN style user names.
You may want to try out JCIFS based NTLM engine and see if it works
better.
> 2) How should I set credentials for NTLM where there is no domain, like
> when using the UPN form? NTCredentials with domain "" or null, or
> UsernamePasswordCredentials? I have tried various combinations
> without success.
>
As far as I understand if it is a standalone workstation the domain
should be same as the the workstation name.
> 3) I am accessing a web service from outside the NT domain so I have no
> workstation name for the 3rd parameter to the NTCredentials
> constructor. I have tried passing null, "", or "somejunk" without
> success.
>
Again, my knowledge of MS Windows networking is awfully outdated, but as
far as I remember there is _always_ an authentication domain of some
sort. In the simplest form the workstation can act as its own auth
domain.
Anyhow it has been ages since I touches anything called Windows
> 4) The NTCredentials constructor seems happy to set workstation to null
> but I find that when it is null a NullPointerException is thrown from
> somewhere within your NTLM code.
>
Feel free to raise a JIRA for this defect.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Full NTLMv2 Support Achieved Easily (Was: NTLM authentication
with a UPN instead of domain and user name)
Posted by Marc Boorshtein <mb...@gmail.com>.
On Thu, Apr 21, 2011 at 5:24 PM, Ron Jacobs
<Ro...@reardencommerce.com> wrote:
> I am not saying that anything doesn't work. I am saying that I have
> looked at the code and see multiple problems based upon my reading of
> the NTLM specification that was first publicly released by Microsoft in
> March of 2007.
>
Fair enough. I was more concerned that MS was allowing something that
shouldn't work :-)
Thanks
Marc
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Full NTLMv2 Support Achieved Easily (Was: NTLM authentication
with a UPN instead of domain and user name)
Posted by Ron Jacobs <Ro...@Reardencommerce.com>.
I am not saying that anything doesn't work. I am saying that I have
looked at the code and see multiple problems based upon my reading of
the NTLM specification that was first publicly released by Microsoft in
March of 2007.
I am still trying to walk on eggshells so as to not be critical and I
will not give specific technical examples. But to reiterate, this code
appears to predate the public release of the NTLM specification.
Although I did find a minor issue in the JCIFS code on the deleted web
page, it was easy to detect and fix and that was the intent of my first
posting earlier today.
If the code you are using is working with the specific configuration
that you are using then I don't see any pressing reason for concern. On
the other hand if you, like I, do not control any of the many variables
inherent in the Windows servers against which you must be able to
authenticate, then you may be interested in the code that I posted.
-----Original Message-----
From: Marc Boorshtein [mailto:mboorshtein@gmail.com]
Sent: Thursday, April 21, 2011 2:14 PM
To: HttpClient User Discussion
Subject: Re: Full NTLMv2 Support Achieved Easily (Was: NTLM authentication with a UPN instead of domain and user name)
I find this discussion fascinating. One question, are you saying the Ntlmschemefactory doesn't work? I use it as part of my unit tests against adfs 2 running on w2k8r2 64bit and it works perfectly.
Thanks
Marc
Sent from my iPad
On Apr 21, 2011, at 5:07 PM, Ron Jacobs <Ro...@Reardencommerce.com> wrote:
> I must say that I hadn't wanted to say anything negative about that code
> in my original post(s), but now that you've asked:
>
> The code now in your 4.1 distribution appears to be minimally (if at
> all) unchanged from some code that I came across during a Google search
> for better supporting NTLM within HttpClient way before I ever upgraded
> to 4.x. I looked at that code in depth sometime last year and concluded
> that there were just too many problems with it.
>
> Without going into technical details, which I have certainly mostly
> forgotten by now anyway, that code seemed to have been written by
> reverse engineering and guessing about NTLM some time before Microsoft
> (finally) publicly released the NTLM specification. It may have worked
> at one time for some specific combination of Windows parameters and
> options but it was too far away from working for the general cases that
> I needed. You see, unlike many NTLM client-side users that are just
> trying to authenticate against a specific Windows server, I need to work
> with just about any combo of Windows OS versions, service packs,
> registry settings, installed apps, etc.
>
> So I abandoned that effort and when I recognized the same code inside
> HttpClient, I was not hopeful. It was as I was looking for alternatives
> that I asked the questions that you answered for me last month leading
> me straight to this approach that is working great for us today.
>
> Seems to me that there is still no "open source" solution that is ready
> to drop into the HttpClient distribution. I believe that the correct
> approach is indeed JCIFS and that your restoring and updating the web
> page is the best solution. If I were "forced" to write some NTLM code
> without licensing issues for HttpClient it would end up looking much
> too uncomfortably close to JCIFS.
>
> I truly hope that I have offended no one.
>
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Thursday, April 21, 2011 12:23 PM
> To: HttpClient User Discussion
> Subject: Re: Full NTLMv2 Support Achieved Easily (Was: NTLM authentication with a UPN instead of domain and user name)
>
> ...
>
> PS: Would you be by any change willing to take a look at the default
> NTLM engine distributed with HttpClient and see what may be wrong there?
> It'd be a great contribution to all users of HttpClient.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Full NTLMv2 Support Achieved Easily (Was: NTLM authentication with a UPN instead of domain and user name)
Posted by Marc Boorshtein <mb...@gmail.com>.
I find this discussion fascinating. One question, are you saying the Ntlmschemefactory doesn't work? I use it as part of my unit tests against adfs 2 running on w2k8r2 64bit and it works perfectly.
Thanks
Marc
Sent from my iPad
On Apr 21, 2011, at 5:07 PM, Ron Jacobs <Ro...@Reardencommerce.com> wrote:
> I must say that I hadn't wanted to say anything negative about that code
> in my original post(s), but now that you've asked:
>
> The code now in your 4.1 distribution appears to be minimally (if at
> all) unchanged from some code that I came across during a Google search
> for better supporting NTLM within HttpClient way before I ever upgraded
> to 4.x. I looked at that code in depth sometime last year and concluded
> that there were just too many problems with it.
>
> Without going into technical details, which I have certainly mostly
> forgotten by now anyway, that code seemed to have been written by
> reverse engineering and guessing about NTLM some time before Microsoft
> (finally) publicly released the NTLM specification. It may have worked
> at one time for some specific combination of Windows parameters and
> options but it was too far away from working for the general cases that
> I needed. You see, unlike many NTLM client-side users that are just
> trying to authenticate against a specific Windows server, I need to work
> with just about any combo of Windows OS versions, service packs,
> registry settings, installed apps, etc.
>
> So I abandoned that effort and when I recognized the same code inside
> HttpClient, I was not hopeful. It was as I was looking for alternatives
> that I asked the questions that you answered for me last month leading
> me straight to this approach that is working great for us today.
>
> Seems to me that there is still no "open source" solution that is ready
> to drop into the HttpClient distribution. I believe that the correct
> approach is indeed JCIFS and that your restoring and updating the web
> page is the best solution. If I were "forced" to write some NTLM code
> without licensing issues for HttpClient it would end up looking much
> too uncomfortably close to JCIFS.
>
> I truly hope that I have offended no one.
>
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Thursday, April 21, 2011 12:23 PM
> To: HttpClient User Discussion
> Subject: Re: Full NTLMv2 Support Achieved Easily (Was: NTLM authentication with a UPN instead of domain and user name)
>
> ...
>
> PS: Would you be by any change willing to take a look at the default
> NTLM engine distributed with HttpClient and see what may be wrong there?
> It'd be a great contribution to all users of HttpClient.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Full NTLMv2 Support Achieved Easily (Was: NTLM authentication
with a UPN instead of domain and user name)
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2011-04-21 at 21:07 +0000, Ron Jacobs wrote:
> I must say that I hadn't wanted to say anything negative about that code
> in my original post(s), but now that you've asked:
>
> The code now in your 4.1 distribution appears to be minimally (if at
> all) unchanged from some code that I came across during a Google search
> for better supporting NTLM within HttpClient way before I ever upgraded
> to 4.x. I looked at that code in depth sometime last year and concluded
> that there were just too many problems with it.
>
> Without going into technical details, which I have certainly mostly
> forgotten by now anyway, that code seemed to have been written by
> reverse engineering and guessing about NTLM some time before Microsoft
> (finally) publicly released the NTLM specification. It may have worked
> at one time for some specific combination of Windows parameters and
> options but it was too far away from working for the general cases that
> I needed. You see, unlike many NTLM client-side users that are just
> trying to authenticate against a specific Windows server, I need to work
> with just about any combo of Windows OS versions, service packs,
> registry settings, installed apps, etc.
>
This sounds quite unsettling. The NTLM engine used by HttpClient by
default was contributed by the Lucene connector framework project as one
big code drop. Since the code came from another ASF project I simply
assumed it was clean and Lucene folks had done their due diligence.
I am less concerned about quality of that code, but I am unpleasantly
surprised that the implementation might have been based on an exiting
piece of code developed through reverse engineering rather than based on
the published specification. The main problem is that HttpClient may now
have a large body of code that was originally published under
uncertain / unclear licensing terms or under license incompatible with
ASLv2.
Is there any chance you could dig out a reference to what you believe is
the original implementation?
> So I abandoned that effort and when I recognized the same code inside
> HttpClient, I was not hopeful. It was as I was looking for alternatives
> that I asked the questions that you answered for me last month leading
> me straight to this approach that is working great for us today.
>
> Seems to me that there is still no "open source" solution that is ready
> to drop into the HttpClient distribution. I believe that the correct
> approach is indeed JCIFS and that your restoring and updating the web
> page is the best solution. If I were "forced" to write some NTLM code
> without licensing issues for HttpClient it would end up looking much
> too uncomfortably close to JCIFS.
>
I reckon no other group of people outside Microsoft know Microsoft
network protocols better than Samba folks. I am sure that their NTLM
implementation is solid and more comprehensive. I wish we could simply
use it instead of having to maintain our own implementation.
The problem is JCIFS is licensed under LGPL which makes its licensing
terms incompatible with non-viral licenses such as ASLv2. As long as you
are using JCIFS for your own needs or for an internal system in your
company you are all fine. However, if you plan to distribute your
application as a commercial product, LGPL becomes an issue and a legal
liability.
> I truly hope that I have offended no one.
>
No, really not. You helped a great deal.
The NTLM guide has been restored and updated based on your input. Please
review:
http://svn.apache.org/viewvc?rev=1095921&view=rev
http://svn.apache.org/viewvc/httpcomponents/httpclient/trunk/src/site/apt/ntlm.apt?view=markup
Cheers
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Full NTLMv2 Support Achieved Easily (Was: NTLM authentication
with a UPN instead of domain and user name)
Posted by Ron Jacobs <Ro...@Reardencommerce.com>.
I must say that I hadn't wanted to say anything negative about that code
in my original post(s), but now that you've asked:
The code now in your 4.1 distribution appears to be minimally (if at
all) unchanged from some code that I came across during a Google search
for better supporting NTLM within HttpClient way before I ever upgraded
to 4.x. I looked at that code in depth sometime last year and concluded
that there were just too many problems with it.
Without going into technical details, which I have certainly mostly
forgotten by now anyway, that code seemed to have been written by
reverse engineering and guessing about NTLM some time before Microsoft
(finally) publicly released the NTLM specification. It may have worked
at one time for some specific combination of Windows parameters and
options but it was too far away from working for the general cases that
I needed. You see, unlike many NTLM client-side users that are just
trying to authenticate against a specific Windows server, I need to work
with just about any combo of Windows OS versions, service packs,
registry settings, installed apps, etc.
So I abandoned that effort and when I recognized the same code inside
HttpClient, I was not hopeful. It was as I was looking for alternatives
that I asked the questions that you answered for me last month leading
me straight to this approach that is working great for us today.
Seems to me that there is still no "open source" solution that is ready
to drop into the HttpClient distribution. I believe that the correct
approach is indeed JCIFS and that your restoring and updating the web
page is the best solution. If I were "forced" to write some NTLM code
without licensing issues for HttpClient it would end up looking much
too uncomfortably close to JCIFS.
I truly hope that I have offended no one.
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Thursday, April 21, 2011 12:23 PM
To: HttpClient User Discussion
Subject: Re: Full NTLMv2 Support Achieved Easily (Was: NTLM authentication with a UPN instead of domain and user name)
...
PS: Would you be by any change willing to take a look at the default
NTLM engine distributed with HttpClient and see what may be wrong there?
It'd be a great contribution to all users of HttpClient.
Re: Full NTLMv2 Support Achieved Easily (Was: NTLM authentication
with a UPN instead of domain and user name)
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2011-04-21 at 17:12 +0000, Ron Jacobs wrote:
> (I apologize for the delay in posting this update. It was delayed due to
> matters outside of my control. Nonetheless, I am sure that anyone
> struggling to gain NTLMv2 support with version 4.1.x will be helped by
> taking a look at the following:)
>
> ------------------------------------------------------------------------
>
> I have tried JCIFS and was able to get all of my test cases to pass.
>
> But, it doesn't support NTLMv2 "out of the box" because its
> NegotiateFlags (see section 2.2.2.5 of the Microsoft's [MS-NLMP] spec)
> is insufficient.
>
> My implementation is based upon the example that used to be found at:
>
> http://hc.apache.org/httpcomponents-client/ntlm.html
>
> I was able to locate the info on the way-back machine at:
>
> http://replay.waybackmachine.org/20090611051135/http://hc.apache.org/httpcomponents-client/ntlm.html
>
> Since the built-in NTLM implementation is insufficient, the maintainers
> may wish to restore this page, with the slight adjustment that I found
> necessary, or with the following code, which I have verified works with
> both NTLMv1 and NTLMv2 authentication:
>
Hi Ron
Many thanks for sharing this information with us. I am going to restore
the document and with your permission update it with the details and
code snippets that you posted in your previous message.
Cheers
Oleg
PS: Would you be by any change willing to take a look at the default
NTLM engine distributed with HttpClient and see what may be wrong there?
It'd be a great contribution to all users of HttpClient.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org