You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@santuario.apache.org by Berin Lautenbach <be...@wingsofhermes.org> on 2004/08/01 14:39:39 UTC
Re: XmlDsig error with base64Binary Type
Ivan,
I've just spent a bit of time playing around with the samples you sent
through.
Two separate issues :
1. Your IAIK sample uses an XPath-Filter that according to my read
should be the same as an enveloping transforms - but for some reason the
xsec library is not outputting attributes when canonicalising. This
looks like a bug in the library, and would explain why the IAIK sample
does not validate.
2. My understanding is that the use of base64binary in a schema will
cause a validating parser to callapse any whitespace during validation.
So (assuming my understanding is correct) Xerces is stripping new
lines (and collapsing spaces) from the base64 data as part of the
initial validation. That's why the output signature has no new lines -
it's not xsec, it's a natural part of validating a base64binary type
element.
Cheers,
Berin
Berin Lautenbach wrote:
> Ivan,
>
> Certainly doesn't sound right! Do you have an example XML file, signed
> with IAIK that I can have a look at?
>
> Cheers,
> Berin
>
> ivan.fontanals@safelayer.com wrote:
>
>>
>> Hi everybody!
>> I'm doing interoperability tests between the IAIK toolkit and
>> the Apache XMLDsig C++, but I've found an error that I'd like to
>> comment with you cause I'm not really sure it was a bug...
>> My scenario is the following:
>> I've got a XML file with its schema. What I'm doing
>> with both toolkits is to generate a signature from a particular XML
>> node with one toolkit and then I try to verify the signature with the
>> other one. This was working fine until I introduced the base64Binary
>> element into the XML schema.
>> What I've realised is that if the XML contains
>> a base64 text identified with "xs:string" in the squema, the signature
>> can be validated. In other words, both toolkits generates the same
>> hash, so it's possible to verify this either with AIAK or XMLDsig
>> toolkit.
>>
>> The problem appears when this element is a "xs:base64Binary" instead
>> of "xs:string"; in that case, AIAK toolkits generates the same hash as
>> it was a xs:string element ( which I think is correct..), but XMLDsig
>> generates a different hash! Is this a bug? Why the hash depends on the
>> schema file?
>>
>> Ah! The hash is different if the base64 text contains carry returns,
>> but is correct if the base64 is written in the same line!
>>
>> Could anybody tell me something about this? I'm using the last
>> realease (xml_sec_1.1) for C++.
>> Thank you very much in advance!
>>
>> Ivan
>>
>>
>> >-<
>> This email has been digitally signed. You can verify its authenticity
>> by installing Safelayer's Root Certificate:
>> http://ca.safelayer.com/install_root.html
>> >-<
>>
>> IMPORTANT NOTICE: This communication contains information which is
>> confidential and may also be privileged. It is for the exclusive use
>> of the intended recipient(s). If you are not the intended recipient(s)
>> please note that any form of distribution, copying or use of this
>> communication or the information in it is strictly prohibited and may
>> be unlawful. If you have received this communication in error please
>> return it to the sender. The opinions expressed within this
>> communication are not necessarily those expressed by Safelayer Secure
>> Communications.
>
>
>
RE: Problems with default namespace and XPath
Posted by iv...@safelayer.com.
Oh! I forgot to tell that I'm working in a C++ environment, using
Xerces_2_5_0 and Xalan_1_8.
Thank you very much anyway, Maarten.
>-<
This email has been digitally signed. You can verify its authenticity by
installing Safelayer's Root Certificate:
http://ca.safelayer.com/install_root.html
>-<
IMPORTANT NOTICE: This communication contains information which is
confidential and may also be privileged. It is for the exclusive use of
the intended recipient(s). If you are not the intended recipient(s) please
note that any form of distribution, copying or use of this communication
or the information in it is strictly prohibited and may be unlawful. If
you have received this communication in error please return it to the
sender. The opinions expressed within this communication are not
necessarily those expressed by Safelayer Secure Communications.
maarten.gerbrands@nc3a.nato.int
07/09/2004 17:09
Please respond to security-dev
To: security-dev@xml.apache.org
cc:
Subject: RE: Problems with default namespace and XPath
I had the same problem using Java 2 SDK, SE v1.4.2_04.
Using the Apache Xerces-j2-2.6.2.jar instead of the default Apache
classes of this SDK I didn't have this error anymore.
I hope this helps solving your problem.
Maarten Gerbrands
Communication Networks Branch,
Communication and Information Systems Division
NATO C3 Agency
-----Original Message-----
From: ivan.fontanals@safelayer.com [mailto:ivan.fontanals@safelayer.com]
Sent: dinsdag 7 september 2004 16:11
To: security-dev@xml.apache.org
Subject: Problems with default namespace and XPath
Hi everybody,
I'm trying to sign a XML document with a non-prefixed namespace,
which is the default one. Well, in fact the problem neither is in the
signature nor in the verification, but is a XPath problem. When I try to
get a DOMNode * from a valid XPath, I always got an error telling that
the node does not exist... The problem is the default namespace, I
think, but I don't know how to solve it.
I'm working with a XML like this:
<?xml version="1.0" encoding="utf-8" standalone="no" ?>
<QUERY xmlns="http://inti.notariado.org/XML"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<REQUEST ID="REQUEST">
<HEADER>AFASFA</HEADER>
</REQUEST>
</QUERY >
If I ask for the /QUERY/REQUEST I got an error, but if I named
the namespace, the problem disappear. I mean this:
<?xml version="1.0" encoding="utf-8" standalone="no" ?>
<ns1:QUERY xmlns:ns1="http://inti.notariado.org/XML"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ns1:REQUEST ID="REQUEST">
<ns1:HEADER>AFASFA</ns1:HEADER>
</ns1:REQUEST>
</ns1:QUERY >
Now I can ask for /ns1:QUERY/ns1:REQUEST without problems, but
this is not what I want.....
Do you know if exists an easy way where I can select a node from
a valid XPath within a default namespace?
My code which returns the node from a XPath is the following:
DOMNode* XMLUtils::getDOMNodeFromXPath( const sfly::string&
xpath ) {
XercesDOMSupport xds;
XalanNode *contextNode;
XercesParserLiaison theParserLiaison;
auto_ptr<XPathEvaluator> theEvaluator( new
XPathEvaluator() );
XalanDocument *theDoc = theParserLiaison.createDocument(
parser->getDocument());
XalanDocumentPrefixResolver thePrefixResolver( theDoc );
XalanDOMString dsPath( xpath.GetData(),
xpath.GetLength() );
const XalanDOMChar *path = dsPath.c_str();
contextNode = theEvaluator->selectSingleNode( xds,
theDoc, path, thePrefixResolver );
return XalanNodeToDOMNode( contextNode );
}
Thank you very much in advance!
Best Regards,
Ivan
>-<
This email has been digitally signed. You can verify its authenticity
by installing Safelayer's Root Certificate:
http://ca.safelayer.com/install_root.html
>-<
IMPORTANT NOTICE: This communication contains information which is
confidential and may also be privileged. It is for the exclusive use of
the intended recipient(s). If you are not the intended recipient(s)
please note that any form of distribution, copying or use of this
communication or the information in it is strictly prohibited and may be
unlawful. If you have received this communication in error please return
it to the sender. The opinions expressed within this communication are
not necessarily those expressed by Safelayer Secure Communications.
RE: Problems with default namespace and XPath
Posted by ma...@nc3a.nato.int.
I had the same problem using Java 2 SDK, SE v1.4.2_04.
Using the Apache Xerces-j2-2.6.2.jar instead of the default Apache
classes of this SDK I didn't have this error anymore.
I hope this helps solving your problem.
Maarten Gerbrands
Communication Networks Branch,
Communication and Information Systems Division
NATO C3 Agency
-----Original Message-----
From: ivan.fontanals@safelayer.com [mailto:ivan.fontanals@safelayer.com]
Sent: dinsdag 7 september 2004 16:11
To: security-dev@xml.apache.org
Subject: Problems with default namespace and XPath
Hi everybody,
I'm trying to sign a XML document with a non-prefixed namespace,
which is the default one. Well, in fact the problem neither is in the
signature nor in the verification, but is a XPath problem. When I try to
get a DOMNode * from a valid XPath, I always got an error telling that
the node does not exist... The problem is the default namespace, I
think, but I don't know how to solve it.
I'm working with a XML like this:
<?xml version="1.0" encoding="utf-8" standalone="no" ?>
<QUERY xmlns="http://inti.notariado.org/XML"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<REQUEST ID="REQUEST">
<HEADER>AFASFA</HEADER>
</REQUEST>
</QUERY >
If I ask for the /QUERY/REQUEST I got an error, but if I named
the namespace, the problem disappear. I mean this:
<?xml version="1.0" encoding="utf-8" standalone="no" ?>
<ns1:QUERY xmlns:ns1="http://inti.notariado.org/XML"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ns1:REQUEST ID="REQUEST">
<ns1:HEADER>AFASFA</ns1:HEADER>
</ns1:REQUEST>
</ns1:QUERY >
Now I can ask for /ns1:QUERY/ns1:REQUEST without problems, but
this is not what I want.....
Do you know if exists an easy way where I can select a node from
a valid XPath within a default namespace?
My code which returns the node from a XPath is the following:
DOMNode* XMLUtils::getDOMNodeFromXPath( const sfly::string&
xpath ) {
XercesDOMSupport xds;
XalanNode *contextNode;
XercesParserLiaison theParserLiaison;
auto_ptr<XPathEvaluator> theEvaluator( new
XPathEvaluator() );
XalanDocument *theDoc = theParserLiaison.createDocument(
parser->getDocument());
XalanDocumentPrefixResolver thePrefixResolver( theDoc );
XalanDOMString dsPath( xpath.GetData(),
xpath.GetLength() );
const XalanDOMChar *path = dsPath.c_str();
contextNode = theEvaluator->selectSingleNode( xds,
theDoc, path, thePrefixResolver );
return XalanNodeToDOMNode( contextNode );
}
Thank you very much in advance!
Best Regards,
Ivan
>-<
This email has been digitally signed. You can verify its authenticity
by installing Safelayer's Root Certificate:
http://ca.safelayer.com/install_root.html
>-<
IMPORTANT NOTICE: This communication contains information which is
confidential and may also be privileged. It is for the exclusive use of
the intended recipient(s). If you are not the intended recipient(s)
please note that any form of distribution, copying or use of this
communication or the information in it is strictly prohibited and may be
unlawful. If you have received this communication in error please return
it to the sender. The opinions expressed within this communication are
not necessarily those expressed by Safelayer Secure Communications.
Problems with default namespace and XPath
Posted by iv...@safelayer.com.
Hi everybody,
I'm trying to sign a XML document with a non-prefixed namespace,
which is the default one. Well, in fact the problem neither is in the
signature nor in the verification, but is a XPath problem. When I try to
get a DOMNode * from a valid XPath, I always got an error telling that the
node does not exist... The problem is the default namespace, I think, but
I don't know how to solve it.
I'm working with a XML like this:
<?xml version="1.0" encoding="utf-8" standalone="no" ?>
<QUERY xmlns="http://inti.notariado.org/XML"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<REQUEST ID="REQUEST">
<HEADER>AFASFA</HEADER>
</REQUEST>
</QUERY >
If I ask for the /QUERY/REQUEST I got an error, but if I named the
namespace, the problem disappear. I mean this:
<?xml version="1.0" encoding="utf-8" standalone="no" ?>
<ns1:QUERY xmlns:ns1="http://inti.notariado.org/XML"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ns1:REQUEST ID="REQUEST">
<ns1:HEADER>AFASFA</ns1:HEADER>
</ns1:REQUEST>
</ns1:QUERY >
Now I can ask for /ns1:QUERY/ns1:REQUEST without problems, but
this is not what I want.....
Do you know if exists an easy way where I can select a node from a
valid XPath within a default namespace?
My code which returns the node from a XPath is the following:
DOMNode* XMLUtils::getDOMNodeFromXPath( const sfly::string& xpath
) {
XercesDOMSupport xds;
XalanNode *contextNode;
XercesParserLiaison theParserLiaison;
auto_ptr<XPathEvaluator> theEvaluator( new
XPathEvaluator() );
XalanDocument *theDoc = theParserLiaison.createDocument(
parser->getDocument());
XalanDocumentPrefixResolver thePrefixResolver( theDoc );
XalanDOMString dsPath( xpath.GetData(), xpath.GetLength()
);
const XalanDOMChar *path = dsPath.c_str();
contextNode = theEvaluator->selectSingleNode( xds, theDoc,
path, thePrefixResolver );
return XalanNodeToDOMNode( contextNode );
}
Thank you very much in advance!
Best Regards,
Ivan
>-<
This email has been digitally signed. You can verify its authenticity by
installing Safelayer's Root Certificate:
http://ca.safelayer.com/install_root.html
>-<
IMPORTANT NOTICE: This communication contains information which is
confidential and may also be privileged. It is for the exclusive use of
the intended recipient(s). If you are not the intended recipient(s) please
note that any form of distribution, copying or use of this communication
or the information in it is strictly prohibited and may be unlawful. If
you have received this communication in error please return it to the
sender. The opinions expressed within this communication are not
necessarily those expressed by Safelayer Secure Communications.
Re: XmlDsig error with base64Binary Type
Posted by Berin Lautenbach <be...@wingsofhermes.org>.
Scott Cantor wrote:
> It's probably worth documenting this better somewhere, though. People
> stumble over this one a lot if they sign base64 data.
Good point. I will do this. (Or at least add to my ever growing list
of things to do :<.)
Cheers,
Berin
RE: XmlDsig error with base64Binary Type
Posted by Scott Cantor <ca...@osu.edu>.
> Do you want me to change anything in Axis? WSS4J? xml-security? to
> support this switch?
I don't think there's anything you can change. Most of the operations in
xmlsec operate on a DOM, at which point it's already been parsed. The
parsing step is where you have to step in and prevent the corruption.
If there's someplace in the code where you do an actual parse operation, I
doubt you're schema validating anyway.
It's probably worth documenting this better somewhere, though. People
stumble over this one a lot if they sign base64 data.
But I'm not really familiar enough with either Axis or WSS4J to know if
there's some kind of obvious change you could make to them.
-- Scott
Re: XmlDsig error with base64Binary Type
Posted by Davanum Srinivas <da...@gmail.com>.
Do you want me to change anything in Axis? WSS4J? xml-security? to
support this switch?
-- dims
On Sun, 8 Aug 2004 13:27:57 -0400, Scott Cantor <ca...@osu.edu> wrote:
> > whitespace-in-element-content? datatype-normalization?
>
> It's datatype-normalization. The other one has to do with something that schema processing overrides.
>
> -- Scott
>
>
--
Davanum Srinivas - http://webservices.apache.org/~dims/
RE: XmlDsig error with base64Binary Type
Posted by Scott Cantor <ca...@osu.edu>.
> whitespace-in-element-content? datatype-normalization?
It's datatype-normalization. The other one has to do with something that schema processing overrides.
-- Scott
Re: XmlDsig error with base64Binary Type
Posted by Davanum Srinivas <da...@gmail.com>.
whitespace-in-element-content? datatype-normalization?
On Tue, 3 Aug 2004 10:00:34 -0400, Scott Cantor <ca...@osu.edu> wrote:
> > Scott indicated that it may be possible to turn off normalisation during
> > the parsing process, but other than that the only other work-around is
> > not to validate during parse.
>
> It is possible with Xerces, we do it in OpenSAML to enable signing a signed
> object, which of course includes base64 data. There's a parser "feature" or
> whatever they call it to disable schema normalization.
>
> -- Scott
>
>
--
Davanum Srinivas - http://webservices.apache.org/~dims/
RE: XmlDsig error with base64Binary Type
Posted by Scott Cantor <ca...@osu.edu>.
> Scott indicated that it may be possible to turn off normalisation during
> the parsing process, but other than that the only other work-around is
> not to validate during parse.
It is possible with Xerces, we do it in OpenSAML to enable signing a signed
object, which of course includes base64 data. There's a parser "feature" or
whatever they call it to disable schema normalization.
-- Scott
Re: XmlDsig error with base64Binary Type
Posted by Berin Lautenbach <be...@wingsofhermes.org>.
Ivan,
As part of validation, a parser will normalise according to pre-defined
rules for each data type. Part of the normalisation process for
base64binary types is to collapse white space down to a single space -
which will break any signature over that data.
Scott indicated that it may be possible to turn off normalisation during
the parsing process, but other than that the only other work-around is
not to validate during parse.
So - to answer your question. If you create a document with a
base64binary element that contains new lines and then sign it (using
either IAIK and xsec), the resulting file will quite possible fail
signature validation under either library if it validated as it is
parsed back into memory. The base64binary normalisation process will
break the signature.
Cheers,
Berin
ivan.fontanals@safelayer.com wrote:
>
> Hi Berin,
> well, first of all I would like to thanks you for all the work
> and tests done!
>
> I 'll ckeck the new CVS version, but I have a last question: With this
> solved bug, will be possible now to validate a XML signed with XSEC
> against IAIK toolkit ?
> I mean if the returned Xsec hash will be the same in both
> toolkits when exists the base64Binary type in the schema....
>
> Again, thank you very much!
>
> Cheers,
> Ivan
> >-<
> This email has been digitally signed. You can verify its authenticity
> by installing Safelayer's Root Certificate:
> http://ca.safelayer.com/install_root.html
> >-<
>
> IMPORTANT NOTICE: This communication contains information which is
> confidential and may also be privileged. It is for the exclusive use of
> the intended recipient(s). If you are not the intended recipient(s)
> please note that any form of distribution, copying or use of this
> communication or the information in it is strictly prohibited and may be
> unlawful. If you have received this communication in error please return
> it to the sender. The opinions expressed within this communication are
> not necessarily those expressed by Safelayer Secure Communications.
>
>
> *Berin Lautenbach <be...@wingsofhermes.org>*
>
> 03/08/2004 13:07
> Please respond to security-dev
>
>
> To: security-dev@xml.apache.org
> cc:
> Subject: Re: XmlDsig error with base64Binary Type
>
>
>
>
> Ivan,
>
> > 1. Your IAIK sample uses an XPath-Filter that according to my read
> > should be the same as an enveloping transforms - but for some reason the
> > xsec library is not outputting attributes when canonicalising. This
> > looks like a bug in the library, and would explain why the IAIK sample
> > does not validate.
>
> There *was* a bug in the library. There was a case where attribute
> nodes would get dropped from the input nodeset that is passed to the
> xpath-filter transform. I've checked a fix into CVS and the CVS code
> now validates the IAIK sample as OK.
>
> Cheers,
> Berin
>
>
Re: XmlDsig error with base64Binary Type
Posted by iv...@safelayer.com.
Hi Berin,
well, first of all I would like to thanks you for all the work and
tests done!
I 'll ckeck the new CVS version, but I have a last question: With this
solved bug, will be possible now to validate a XML signed with XSEC
against IAIK toolkit ?
I mean if the returned Xsec hash will be the same in both
toolkits when exists the base64Binary type in the schema....
Again, thank you very much!
Cheers,
Ivan
>-<
This email has been digitally signed. You can verify its authenticity by
installing Safelayer's Root Certificate:
http://ca.safelayer.com/install_root.html
>-<
IMPORTANT NOTICE: This communication contains information which is
confidential and may also be privileged. It is for the exclusive use of
the intended recipient(s). If you are not the intended recipient(s) please
note that any form of distribution, copying or use of this communication
or the information in it is strictly prohibited and may be unlawful. If
you have received this communication in error please return it to the
sender. The opinions expressed within this communication are not
necessarily those expressed by Safelayer Secure Communications.
Berin Lautenbach <be...@wingsofhermes.org>
03/08/2004 13:07
Please respond to security-dev
To: security-dev@xml.apache.org
cc:
Subject: Re: XmlDsig error with base64Binary Type
Ivan,
> 1. Your IAIK sample uses an XPath-Filter that according to my read
> should be the same as an enveloping transforms - but for some reason the
> xsec library is not outputting attributes when canonicalising. This
> looks like a bug in the library, and would explain why the IAIK sample
> does not validate.
There *was* a bug in the library. There was a case where attribute
nodes would get dropped from the input nodeset that is passed to the
xpath-filter transform. I've checked a fix into CVS and the CVS code
now validates the IAIK sample as OK.
Cheers,
Berin
Re: XmlDsig error with base64Binary Type
Posted by Berin Lautenbach <be...@wingsofhermes.org>.
Ivan,
> 1. Your IAIK sample uses an XPath-Filter that according to my read
> should be the same as an enveloping transforms - but for some reason the
> xsec library is not outputting attributes when canonicalising. This
> looks like a bug in the library, and would explain why the IAIK sample
> does not validate.
There *was* a bug in the library. There was a case where attribute
nodes would get dropped from the input nodeset that is passed to the
xpath-filter transform. I've checked a fix into CVS and the CVS code
now validates the IAIK sample as OK.
Cheers,
Berin
Re: XmlDsig error with base64Binary Type
Posted by Berin Lautenbach <be...@wingsofhermes.org>.
Scott Cantor wrote:
>>Or I suppose the other option would be to make sure you validate prior
>>to sign. If I could think of some fancy way to do it, we could even
>>mark base64binary elements somehow and have the security library convert
>>them to normalised form as part of signing.
>
>
> You mean verify first and then validate? You can, but until recently it
> required two parse operations, which isn't reasonable. I don't know if
> Xerces-C supports in-memory validation of a DOM yet. The Java code does.
I was actually thinking more about always making sure the original
document is validated prior to signing, just to make sure you are
signing the normalised form. But that's going to be unworkable in many
instances - particularly if you are building the DOM in memory. It was
just idle musings :>.
I had a quick look at validating in memory with Xerces-C a few months
back and couldn't find anything, but then I wasn't looking too hard.
CHeers,
Berin
RE: XmlDsig error with base64Binary Type
Posted by Scott Cantor <ca...@osu.edu>.
> Or I suppose the other option would be to make sure you validate prior
> to sign. If I could think of some fancy way to do it, we could even
> mark base64binary elements somehow and have the security library convert
> them to normalised form as part of signing.
You mean verify first and then validate? You can, but until recently it
required two parse operations, which isn't reasonable. I don't know if
Xerces-C supports in-memory validation of a DOM yet. The Java code does.
-- Scott
Re: XmlDsig error with base64Binary Type
Posted by Berin Lautenbach <be...@wingsofhermes.org>.
Scott Cantor wrote:
>>2. My understanding is that the use of base64binary in a schema will
>>cause a validating parser to callapse any whitespace during validation.
>
>
> This is true as long as data type normalization is turned on. Most parsers
> let you turn it off. This is required to verify signatures over base64
> elements when validating.
Or I suppose the other option would be to make sure you validate prior
to sign. If I could think of some fancy way to do it, we could even
mark base64binary elements somehow and have the security library convert
them to normalised form as part of signing.
>
> This also sucks, but it's the only workaround if you want to validate. Most
> people don't, and Xerces itself has so many schema-related bugs going
> unfixed lately that it's becoming nearly impossible to keep doing it as an
> implementer.
:<.
Cheers,
Berin
RE: XmlDsig error with base64Binary Type
Posted by Scott Cantor <ca...@osu.edu>.
> 2. My understanding is that the use of base64binary in a schema will
> cause a validating parser to callapse any whitespace during validation.
This is true as long as data type normalization is turned on. Most parsers
let you turn it off. This is required to verify signatures over base64
elements when validating.
This also sucks, but it's the only workaround if you want to validate. Most
people don't, and Xerces itself has so many schema-related bugs going
unfixed lately that it's becoming nearly impossible to keep doing it as an
implementer.
-- Scott