You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Aidan Skinner <ai...@apache.org> on 2008/06/03 18:00:00 UTC
Random .Net question
So, I'm trying to fix QPID-1058 and add CRAM-MD5-HASHED support to the
.Net client. I'm having a bit of a problem, which I think is due to
the signed vs unsigned nature of byte in java and C#.
If I force the key bytes to be within the range 0-127 all is well, but
if they go higher of lower then I start getting different outputs from
the HMAC, which is a bit perturbing.
Don't suppouse anybodys come across something similar before have they?
- Aidan
--
aim/y!:aidans42 g:aidan.skinner@gmail.com
http://aidan.skinner.me.uk/
"We belong to nobody and nobody belongs to us. We don't even belong to
each other."
Re: Random .Net question
Posted by Aidan Skinner <ai...@apache.org>.
On Thu, Jun 5, 2008 at 12:34 PM, Robert Greig <ro...@gmail.com> wrote:
> 2008/6/5 Aidan Skinner <ai...@apache.org>:
>
>> I am clearly losing it, but this program:
>
> This isn't quite the same as the Java. Here is my version:
>> byte[] key =
>> System.Text.UTF8Encoding.UTF8.GetBytes(System.Text.UTF8Encoding.UTF8.GetString(hexkey));
>
> What are you expecting this to do? Return the original byte array? Is
> the original byte array a valid UTF-8 encoding?
I was hoping for the same array as the Java equivalent, the origional
byte array has some characters in the gappy bits of UTF8 (such as
0x8b) and is probably invalid as a result.
- Aidan
--
aim/y!:aidans42 g:aidan.skinner@gmail.com
http://aidan.skinner.me.uk/
"We belong to nobody and nobody belongs to us. We don't even belong to
each other."
Re: Random .Net question
Posted by Robert Greig <ro...@gmail.com>.
2008/6/5 Aidan Skinner <ai...@apache.org>:
> I am clearly losing it, but this program:
This isn't quite the same as the Java. Here is my version:
char[] hexkey = new char[]{(char) 0x08, (char)0x4e, (char)0x03, (char)0x43,
(char)0xa0, (char)0x48, (char)0x6f, (char)0xf0,
(char)0x55, (char)0x30, (char)0xdf,
(char)0x6c,
(char)0x70, (char)0x5c, (char)0x8b,
(char)0xb4};
UTF8Encoding encoding = new UTF8Encoding();
byte[] key = encoding.GetBytes(hexkey);
Console.Out.WriteLine("Key: {0}", HexToString(key));
HMAC hmac = new HMACMD5(key);
byte[] value =
encoding.GetBytes("<16...@localhost>");
Console.Out.WriteLine("Value: {0}", HexToString(value));
byte[] result = hmac.ComputeHash(value);
Console.Out.WriteLine("Values: {0}", HexToString(result));
> byte[] hexkey = new byte[]{0x08, 0x4e, 0x03, 0x43,
> 0xa0, 0x48, 0x6f, 0xf0,
> 0x55, 0x30, 0xdf, 0x6c,
> 0x70, 0x5c, 0x8b, 0xb4};
>
> byte[] key =
> System.Text.UTF8Encoding.UTF8.GetBytes(System.Text.UTF8Encoding.UTF8.GetString(hexkey));
What are you expecting this to do? Return the original byte array? Is
the original byte array a valid UTF-8 encoding?
RG
Re: Random .Net question
Posted by Aidan Skinner <ai...@apache.org>.
On Thu, Jun 5, 2008 at 10:19 AM, Aidan Skinner <ai...@apache.org> wrote:
> On Thu, Jun 5, 2008 at 1:01 AM, Tomas Restrepo
> <to...@devdeo.com> wrote:
>
>> As far as I remember, the .NET client already uses UTF-8:
>>
>> byte[] secret = Encoding.UTF8.GetBytes(passwd);
>>
>> But maybe I'm just missing something. I definitely haven't done as
>> many tests as Aidan :)
>
> It does, this is a new CRAM-MD5-HASHED implementation, the regular
> CRAM-MD5 wouldn't exhibit this with most 'normal'[1] passwords.
I am clearly losing it, but this program:
using System;
using System.Globalization;
namespace Byter
{
class Program
{
static void Main(string[] args)
{
byte[] hexkey = new byte[]{0x08, 0x4e, 0x03, 0x43,
0xa0, 0x48, 0x6f, 0xf0,
0x55, 0x30, 0xdf, 0x6c,
0x70, 0x5c, 0x8b, 0xb4};
byte[] key =
System.Text.UTF8Encoding.UTF8.GetBytes(System.Text.UTF8Encoding.UTF8.GetString(hexkey));
int i = 0;
foreach (byte b in key)
{
Console.Write("0x"+b.ToString("x2",
CultureInfo.InvariantCulture)+",");
if (++i % 4 == 0) {
Console.WriteLine();
}
}
}
}
}
Produces
0x08,0x4e,0x03,0x43,
0x48,0x6f,0x55,0x30,
0x6c,0x70,0x5c,
Which is different from the java equivalent posted previously. I'm
making a note here, HUGE SUCESS.
- Aidan
--
aim/y!:aidans42 g:aidan.skinner@gmail.com
http://aidan.skinner.me.uk/
"We belong to nobody and nobody belongs to us. We don't even belong to
each other."
Re: Random .Net question
Posted by Aidan Skinner <ai...@apache.org>.
On Thu, Jun 5, 2008 at 1:01 AM, Tomas Restrepo
<to...@devdeo.com> wrote:
> As far as I remember, the .NET client already uses UTF-8:
>
> byte[] secret = Encoding.UTF8.GetBytes(passwd);
>
> But maybe I'm just missing something. I definitely haven't done as
> many tests as Aidan :)
It does, this is a new CRAM-MD5-HASHED implementation, the regular
CRAM-MD5 wouldn't exhibit this with most 'normal'[1] passwords.
- Aidan
[1] Showing my Anglo-centricism there, by 'normal' I mean in the
regular 7-bit ASCII range
--
aim/y!:aidans42 g:aidan.skinner@gmail.com
http://aidan.skinner.me.uk/
"We belong to nobody and nobody belongs to us. We don't even belong to
each other."
Re: Random .Net question
Posted by Tomas Restrepo <to...@devdeo.com>.
Rob,
> Yep - looks like the UTF-8 encoding is correct (
> http://tools.ietf.org/html/draft-ietf-sasl-crammd5-09 for the full
> proposal)...
>
> Learn something every day....
>
> Guess the onus is on us to get the .net doing the right thing then...
As far as I remember, the .NET client already uses UTF-8:
byte[] secret = Encoding.UTF8.GetBytes(passwd);
But maybe I'm just missing something. I definitely haven't done as
many tests as Aidan :)
--
Tomas Restrepo
http://winterdom.com/weblog/
http://www.devdeo.com/
Re: Random .Net question
Posted by Robert Godfrey <ro...@gmail.com>.
2008/6/4 Robert Greig <ro...@gmail.com>:
> 2008/6/4 Robert Greig <ro...@gmail.com>:
> >
> >> I've attached a small snippet that I'm going to work into a proper
> >> test case for when I file the bug, the output should be
> >> ff724187be5f9d4df6fd93a7dce660c9
> >
> > In your sample program, why do you convert your password into UTF-8
> > bytes - which makes your byte array 21 bytes long due to the
> > particular chars you chose?
> >
> > If you encode into an 8 bit characterset like ISO-8859-1 you get the
> > result you expect.
>
> My turn to reply to myself - UTF-8 is apparently the recommended
> approach now for encoding
> (http://www.imc.org/ietf-sasl/mail-archive/msg00517.html).
>
Yep - looks like the UTF-8 encoding is correct (
http://tools.ietf.org/html/draft-ietf-sasl-crammd5-09 for the full
proposal)...
Learn something every day....
Guess the onus is on us to get the .net doing the right thing then...
-- Rob
Re: Random .Net question
Posted by Robert Greig <ro...@gmail.com>.
2008/6/4 Robert Greig <ro...@gmail.com>:
>
>> I've attached a small snippet that I'm going to work into a proper
>> test case for when I file the bug, the output should be
>> ff724187be5f9d4df6fd93a7dce660c9
>
> In your sample program, why do you convert your password into UTF-8
> bytes - which makes your byte array 21 bytes long due to the
> particular chars you chose?
>
> If you encode into an 8 bit characterset like ISO-8859-1 you get the
> result you expect.
My turn to reply to myself - UTF-8 is apparently the recommended
approach now for encoding
(http://www.imc.org/ietf-sasl/mail-archive/msg00517.html).
So I would argue that the 21 byte array is right and that the output
should not be as you suggest above?
RG
Re: Random .Net question
Posted by Robert Greig <ro...@gmail.com>.
2008/6/4 Aidan Skinner <ai...@apache.org>:
> I've attached a small snippet that I'm going to work into a proper
> test case for when I file the bug, the output should be
> ff724187be5f9d4df6fd93a7dce660c9
In your sample program, why do you convert your password into UTF-8
bytes - which makes your byte array 21 bytes long due to the
particular chars you chose?
If you encode into an 8 bit characterset like ISO-8859-1 you get the
result you expect.
RG
Re: Random .Net question
Posted by Aidan Skinner <ai...@apache.org>.
On Wed, Jun 4, 2008 at 5:26 PM, Robert Greig <ro...@gmail.com> wrote:
> 2008/6/4 Aidan Skinner <ai...@apache.org>:
>
>> The hash for guest goes in as:
>
> Is this supposed to be digest that is the last 16 bytes of the client
> response? (i.e. the bit after the username and space).
No, this is the digest of the password that's used as the key to the
HMAC to create that digest.
I've attached a small snippet that I'm going to work into a proper
test case for when I file the bug, the output should be
ff724187be5f9d4df6fd93a7dce660c9
In the meantime, i'm going to implement Digest-MD5 instead since that
has knobs to twiddle that control the password character encoding.
- Aidan
--
aim/y!:aidans42 g:aidan.skinner@gmail.com
http://aidan.skinner.me.uk/
"We belong to nobody and nobody belongs to us. We don't even belong to
each other."
Re: Random .Net question
Posted by Robert Greig <ro...@gmail.com>.
2008/6/4 Aidan Skinner <ai...@apache.org>:
> The hash for guest goes in as:
Is this supposed to be digest that is the last 16 bytes of the client
response? (i.e. the bit after the username and space).
> 0x08, 0x4e, 0x03, 0x43,
> 0xa0, 0x48, 0x6f, 0xf0,
> 0x55, 0x30, 0xdf, 0x6c,
> 0x70, 0x5c, 0x8b, 0xb4
RG
Re: Random .Net question
Posted by Aidan Skinner <ai...@apache.org>.
On Wed, Jun 4, 2008 at 3:21 PM, Aidan Skinner <ai...@apache.org> wrote:
> On Wed, Jun 4, 2008 at 2:11 PM, Aidan Skinner <ai...@apache.org> wrote:
Replying to myself this much is clearly a sign of madness.
>> The classes do when used directly, but not when called from the SASL
>> CRAM-MD5 implementation. I've been digging through the OpenJDK source
> I tried the following bit of python as a tie breaker:
> And it produces the same answer as the .Net implementation, and
> differs from the brokers implementation.
The java impl of SASL is broken. It converst the char[] to a String,
then calls getBytes(), which is clearly insane and *adds data*.
The hash for guest goes in as:
0x08, 0x4e, 0x03, 0x43,
0xa0, 0x48, 0x6f, 0xf0,
0x55, 0x30, 0xdf, 0x6c,
0x70, 0x5c, 0x8b, 0xb4
and comes out as:
0x08,0x4e,0x03,0x43,
0xc2,0xa0,0x48,0x6f,
0xc3,0xb0,0x55,0x30,
0xc3,0x9f,0x6c,0x70,
0x5c,0xc2,0x8b,0xc2,
0xb4
Which not only inserts data, but munges bits of it as well.
- Aidan
--
aim/y!:aidans42 g:aidan.skinner@gmail.com
http://aidan.skinner.me.uk/
"We belong to nobody and nobody belongs to us. We don't even belong to
each other."
Re: Random .Net question
Posted by Aidan Skinner <ai...@apache.org>.
On Wed, Jun 4, 2008 at 2:11 PM, Aidan Skinner <ai...@apache.org> wrote:
> The classes do when used directly, but not when called from the SASL
> CRAM-MD5 implementation. I've been digging through the OpenJDK source
> but haven't yet come up with a reason why. I suspect we're doing
> something broken somewhere in our implementation, possibly to do with
> collapsing or not collapsing keys > 64 bytes.
I tried the following bit of python as a tie breaker:
#!/usr/bin/python
import hmac
password = chr(0x08)+chr(0x4e)+chr(0x03)+chr(0x43)+chr(0xa0)+chr(0x48)+chr(0x6f)+chr(0xf0)+chr(0x55)+chr(0x30)+chr(0xdf)+chr(0x6c)+chr(0x70)+chr(0x5c)+chr(0x8b)+chr(0xb4)
print password
challenge = "<-5...@FOOCORP.COM>"
print hmac.HMAC(password, challenge).hexdigest()
And it produces the same answer as the .Net implementation, and
differs from the brokers implementation.
Joy to the world.
- Aidan
--
aim/y!:aidans42 g:aidan.skinner@gmail.com
http://aidan.skinner.me.uk/
"We belong to nobody and nobody belongs to us. We don't even belong to
each other."
Re: Random .Net question
Posted by Aidan Skinner <ai...@apache.org>.
On Wed, Jun 4, 2008 at 2:02 PM, Robert Greig <ro...@gmail.com> wrote:
> 2008/6/4 Aidan Skinner <ai...@apache.org>:
>
>> Interestingly, hardcoding the password to the key used in test case 3
>> also fails. &'ing the password with 0x7f on both sides allows it to
>> work...
>
> What exactly isn't returning what you expect? Are you testing that the
> HMACMD5 class in .NET returns the same MAC as the corresponding Mac
> class in Java for a given key and value?
The classes do when used directly, but not when called from the SASL
CRAM-MD5 implementation. I've been digging through the OpenJDK source
but haven't yet come up with a reason why. I suspect we're doing
something broken somewhere in our implementation, possibly to do with
collapsing or not collapsing keys > 64 bytes.
- Aidan
--
aim/y!:aidans42 g:aidan.skinner@gmail.com
http://aidan.skinner.me.uk/
"We belong to nobody and nobody belongs to us. We don't even belong to
each other."
Re: Random .Net question
Posted by Robert Greig <ro...@gmail.com>.
2008/6/4 Aidan Skinner <ai...@apache.org>:
> Interestingly, hardcoding the password to the key used in test case 3
> also fails. &'ing the password with 0x7f on both sides allows it to
> work...
What exactly isn't returning what you expect? Are you testing that the
HMACMD5 class in .NET returns the same MAC as the corresponding Mac
class in Java for a given key and value?
RG
Re: Random .Net question
Posted by Aidan Skinner <ai...@apache.org>.
On Tue, Jun 3, 2008 at 6:20 PM, Aidan Skinner <ai...@apache.org> wrote:
> Yeah, I tried that as well. It's possible there's something funky
> going on in the java SASL provider I guess (which would be deeply
> unfortunate).
Interestingly, hardcoding the password to the key used in test case 3
also fails. &'ing the password with 0x7f on both sides allows it to
work...
- Aidan
--
aim/y!:aidans42 g:aidan.skinner@gmail.com
http://aidan.skinner.me.uk/
"We belong to nobody and nobody belongs to us. We don't even belong to
each other."
Re: Random .Net question
Posted by Aidan Skinner <ai...@apache.org>.
On Tue, Jun 3, 2008 at 5:37 PM, Tomas Restrepo
<to...@devdeo.com> wrote:
> Weird. A quick check of HMACMD5 against one of the test-cases in
> RFC2202 with key bytes above 0x7F on dotnet seems to yield the right
> results (just tried test3 quickly). Does it return the right result on
> the Java code as well? Just curious
Yeah, I tried that as well. It's possible there's something funky
going on in the java SASL provider I guess (which would be deeply
unfortunate).
I'm going to have some food and think about this for a bit, I've
attached a patch to QPID-1085 with where I'm at just now.
--
aim/y!:aidans42 g:aidan.skinner@gmail.com
http://aidan.skinner.me.uk/
"We belong to nobody and nobody belongs to us. We don't even belong to
each other."
Re: Random .Net question
Posted by Tomas Restrepo <to...@devdeo.com>.
Hi Aidan,
> They bytes are generated from the MD5 of the password, which returns
> the same hex on both platforms. The problem comes when that byte[] is
> passed to the HMACMD5 constructor as the key.
>
> This works (ie. produces the same result as it's java cousin)
>
> byte[] key = new byte[]{0x7F};
> HMAC hmac = new HMACMD5(key);
> byte[] value = hmac.ComputeHash("foo");
>
> Changing that 0x7F to 0x80 does not.
Weird. A quick check of HMACMD5 against one of the test-cases in
RFC2202 with key bytes above 0x7F on dotnet seems to yield the right
results (just tried test3 quickly). Does it return the right result on
the Java code as well? Just curious
--
Tomas Restrepo
http://winterdom.com/weblog/
http://www.devdeo.com/
Re: Random .Net question
Posted by Aidan Skinner <ai...@apache.org>.
On Tue, Jun 3, 2008 at 5:16 PM, Tomas Restrepo
<to...@devdeo.com> wrote:
>> So, I'm trying to fix QPID-1058 and add CRAM-MD5-HASHED support to the
>> .Net client. I'm having a bit of a problem, which I think is due to
>> the signed vs unsigned nature of byte in java and C#.
>>
>> If I force the key bytes to be within the range 0-127 all is well, but
>> if they go higher of lower then I start getting different outputs from
>> the HMAC, which is a bit perturbing.
>>
>> Don't suppouse anybodys come across something similar before have they?
>
> Might this be an encoding issue? I.e. how are the bytes for
> calculating the initial hash derived from the plain-text password
> provided by the caller? The encoding should certainly match on both
> sides, otherwise the byte-representation will differ and the hash
> won't match.... Just a thought.
They bytes are generated from the MD5 of the password, which returns
the same hex on both platforms. The problem comes when that byte[] is
passed to the HMACMD5 constructor as the key.
This works (ie. produces the same result as it's java cousin)
byte[] key = new byte[]{0x7F};
HMAC hmac = new HMACMD5(key);
byte[] value = hmac.ComputeHash("foo");
Changing that 0x7F to 0x80 does not.
- Aidan
--
aim/y!:aidans42 g:aidan.skinner@gmail.com
http://aidan.skinner.me.uk/
"We belong to nobody and nobody belongs to us. We don't even belong to
each other."
Re: Random .Net question
Posted by Tomas Restrepo <to...@devdeo.com>.
Aidan,
> So, I'm trying to fix QPID-1058 and add CRAM-MD5-HASHED support to the
> .Net client. I'm having a bit of a problem, which I think is due to
> the signed vs unsigned nature of byte in java and C#.
>
> If I force the key bytes to be within the range 0-127 all is well, but
> if they go higher of lower then I start getting different outputs from
> the HMAC, which is a bit perturbing.
>
> Don't suppouse anybodys come across something similar before have they?
Might this be an encoding issue? I.e. how are the bytes for
calculating the initial hash derived from the plain-text password
provided by the caller? The encoding should certainly match on both
sides, otherwise the byte-representation will differ and the hash
won't match.... Just a thought.
--
Tomas Restrepo
http://winterdom.com/weblog/
http://www.devdeo.com/