You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@santuario.apache.org by ra...@ralphholz.de on 2007/12/05 20:17:28 UTC
Signature breaks during I/O and parsing, but cannot resolve with canonicalisation
Hi,
I create a signature with a C14N transform applied. Signing works. The
signature is valid if I apply the checkSignatureValue() method directly on
the result Document object.
However, if I write the Document out to the file system and parse it in back
again later, the signature is invalid. So I canonicalise the Document again:
Canonicalizer c14n = Canonicalizer.getInstance(
"http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments");
byte[] outputBytes = c14n.canonicalizeSubtree(document);
And build a fresh one from the bytes:
DocumentBuilderFactory dfactory = DocumentBuilderFactory.newInstance();
dfactory.setNamespaceAware(true);
DocumentBuilder documentBuilder = dfactory.newDocumentBuilder();
Document doc = documentBuilder.parse(new ByteArrayInputStream(outputBytes));
And do
String BaseURI = "";
signature = new XMLSignature(sigElement, BaseURI);
verified = signature.checkSignatureValue(myPublicKey);
However, the signature is still not valid. My guess is this happens because I
use a parser to build the Document from the bytes - which is the standard
javax.xml.parsers from JDK 1.5.
Looking at my document with kdiff3, I can see that only one line is different:
Original signed XML is
<env:Body xmlns:pdpa="http://da.ralphholz.de/PDP-A_1"
xmlns:pdpc="http://da.ralphholz.de/PDP-C"
xmlns:env="http://www.w3.org/2003/05/soap-envelope">
Read-in and canonicalised XML is
<env:Body xmlns:pdpa="http://da.ralphholz.de/PDP-A_1"
xmlns:pdpc="http://da.ralphholz.de/PDP-C">
Although this should not matter because only nodes further down the tree are
signed, but not env:Body.
Where am I going wrong?
Thanks,
Ralph
PS:
The code for write out/read in is the following, although it should not
matter:
<write out>
File file = new File(filename);
FileOutputStream f = new FileOutputStream(file);
TransformerFactory factory = TransformerFactory.newInstance();
Transformer transformer = factory.newTransformer();
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION,
"yes");
transformer.setOutputProperty("indent", "yes");
DOMSource source = new DOMSource(document);
StreamResult result = new StreamResult(f);
transformer.transform(source, result);
f.close();
</write out>
<read in>
dbfFactory.setNamespaceAware(true);
dbfFactory.setValidating(false);
dbfFactory.setAttribute("http://xml.org/sax/features/namespaces",
Boolean.TRUE);
builder = dbfFactory.newDocumentBuilder();
File f = new File(filename);
Document doc = builder.parse(new java.io.FileInputStream(f));
</read in>
--
For contact details, please see www.ralphholz.de.
Re: Signature breaks during I/O and parsing,
but cannot resolve with canonicalisation
Posted by Sean Mullan <Se...@Sun.COM>.
ralph-xmlsecurity@ralphholz.de wrote:
> Hi,
>
> I create a signature with a C14N transform applied. Signing works. The
> signature is valid if I apply the checkSignatureValue() method directly on
> the result Document object.
>
> However, if I write the Document out to the file system and parse it in back
> again later, the signature is invalid.
You need to dump the reference's pre-digested, canonicalized input
during the signing and validating. Then it is a matter of comparing them
to see what is different.
I can't tell what your problem is without seeing the Document contents,
but one problem I have seen a few times are due to signing legacy XML
that has no namespace information, i.e:
<ds:Object>
<Foo>
</Foo>
</ds:Object>
When you canonicalize this, it doesn't see any namespace attributes on
the Foo element and so it inherits the xmldsig namespace. But when you
serialize the Document and deserialize it, xerces or other parsing
software may add an xmlns attribute set to the empty string indicating
that it has no namespace, ex:
<ds:Object>
<Foo xmlns="">
</Foo>
</ds:Object>
which will break the signature.
I don't really know what the proper way to fix this is, but one solution
(if possible) is to always make sure every Element is defined in some
namespace (ex: add an xmlns="" attribute to the Foo element before signing):
<Foo xmlns="">
</Foo>
--Sean
Re: Signature breaks during I/O and parsing, but cannot resolve with
canonicalisation
Posted by Brent Putman <pu...@georgetown.edu>.
Brent Putman wrote:
>
>
> Scott Cantor wrote:
>
>
> I was just about to send this out, but Scott beat me to it. :-)
> Here's where that code comes from:
>
> http://svn.middleware.georgetown.edu/view/trunk/src/org/opensaml/xml/util/XMLHelper.java?root=java-xmltooling&view=markup
>
>
>
>
> These 2 methods are by and large how we serialize DOM elements in the
> Java version of OpenSAML, and seems to work quite successfully with
> signature and encryption.
BTW, we have a pretty print method there too, very similar to what you
were doing, but we *never* use that for anything that's going to get
machine processed (e.g. signature validation), it's only for dumping
stuff out for humans to read, e.g for logging or for dumping to the
console for debugging.
Re: Signature breaks during I/O and parsing, but cannot resolve with
canonicalisation
Posted by Brent Putman <pu...@georgetown.edu>.
Scott Cantor wrote:
>
>> You don't happen to have a code snippet around that does exactly that
>> (Java)?
>>
>
> I don't do Java any more (see headache above). The xmltooling code underneath opensaml-j, which I didn’t write, has helper routines that provide one way to do it, and they appear to use a DOM3 LSSerializer:
>
> public static void writeNode(Node node, Writer output) {
> DOMImplementation domImpl = node.getOwnerDocument().getImplementation();
> DOMImplementationLS domImplLS = (DOMImplementationLS) domImpl.getFeature("LS", "3.0");
> LSSerializer serializer = domImplLS.createLSSerializer();
> LSOutput serializerOut = domImplLS.createLSOutput();
> serializerOut.setCharacterStream(output);
> serializer.write(node, serializerOut);
> }
>
> There's also a pretty-print version that seems to be similar to what you're doing but not exactly the same.
>
I was just about to send this out, but Scott beat me to it. :-) Here's
where that code comes from:
http://svn.middleware.georgetown.edu/view/trunk/src/org/opensaml/xml/util/XMLHelper.java?root=java-xmltooling&view=markup
It's often used via the nodeToString():
public static String nodeToString(Node node) {
StringWriter writer = new StringWriter();
writeNode(node, writer);
return writer.toString();
}
These 2 methods are by and large how we serialize DOM elements in the
Java version of OpenSAML, and seems to work quite successfully with
signature and encryption.
--Brent
RE: Signature breaks during I/O and parsing, but cannot resolve with canonicalisation
Posted by Scott Cantor <ca...@osu.edu>.
> I think these things that I attempted are not too rare - having the XML go
> through different parsers and outputters. Transferring objects between VMs
> might also be something that people want to do, so I am not too sure if it
> is a good idea to check equality of namespaces by object reference.
I don't either, but I haven't followed the discussions and Java strings give me a headache.
> You don't happen to have a code snippet around that does exactly that
> (Java)?
I don't do Java any more (see headache above). The xmltooling code underneath opensaml-j, which I didn’t write, has helper routines that provide one way to do it, and they appear to use a DOM3 LSSerializer:
public static void writeNode(Node node, Writer output) {
DOMImplementation domImpl = node.getOwnerDocument().getImplementation();
DOMImplementationLS domImplLS = (DOMImplementationLS) domImpl.getFeature("LS", "3.0");
LSSerializer serializer = domImplLS.createLSSerializer();
LSOutput serializerOut = domImplLS.createLSOutput();
serializerOut.setCharacterStream(output);
serializer.write(node, serializerOut);
}
There's also a pretty-print version that seems to be similar to what you're doing but not exactly the same.
-- Scott
Re: Signature breaks during I/O and parsing, but cannot resolve with canonicalisation
Posted by ra...@ralphholz.de.
Hi Scott,
thanks for all that information, it is much appreciated. I can see we still
have to go a long way with DSig. Against this background, I have switched
back to an alternative way of signing for now. I think I cannot get it to run
with XMLSec in the time that I have left on this project - but the
en/decryption, at least, works.
I think these things that I attempted are not too rare - having the XML go
through different parsers and outputters. Transferring objects between VMs
might also be something that people want to do, so I am not too sure if it is
a good idea to check equality of namespaces by object reference.
> > It's the identity transform - that should not do anything except store
> > the DOMSource in a Result.
>
> Not if you're pretty printing with some kind of indent flag. That can't
> possibly do what it says if it's really an identity transform....
Yeah, well, as I said, whether I set the indent flag or not does not affect
the outcome.
> > In essence, what I need is to transfer the XML between sender and
> > receiver. What kind of serialisation would you recommend for the
> > transfer?
>
> I definitely wouldn't use code that was likely to affect the DOM in any
> way. I would simply write out the DOM node by node using a serializer that
> was designed to do nothing else to it. Namespaces are another matter...it's
You don't happen to have a code snippet around that does exactly that (Java)?
Ralph
--
For contact details, please see www.ralphholz.de.
RE: Signature breaks during I/O and parsing, but cannot resolve with canonicalisation
Posted by Scott Cantor <ca...@osu.edu>.
> It's the identity transform - that should not do anything except store the
> DOMSource in a Result.
Not if you're pretty printing with some kind of indent flag. That can't possibly do what it says if it's really an identity transform....
> Isn't the whole idea of XML Signature and Canonicalisation that exactly this
> should not happen?
Yes, but they don't work well, and it very much depends on namespace handling and the specifics of your document whether you'll break something by running it through a c14n step yourself. That can affect the namespaces in a destructive manner depending on the c14n the signature or reference(s) are using. Many serializers also affect namespaces in odd ways.
> When I hand XML over to some HTTP agent, there are almost
> certainly small changes (linefeeds etc.) that happen on the way to the
> destination.
If you change the whitespace, you're dead. None of the standard c14n algorithms will handle that. Whitespace is significant in XML, so it *will* change the digest. This is often surprising to people, but it's true. If you have some way to recover the original XML by knowing what whitespace was added or removed, then sure, you could deal with it, but in effect that's like having a new c14n or transform algorithm.
> In essence, what I need is to transfer the XML between sender and receiver.
> What kind of serialisation would you recommend for the transfer?
I definitely wouldn't use code that was likely to affect the DOM in any way. I would simply write out the DOM node by node using a serializer that was designed to do nothing else to it. Namespaces are another matter...it's often necessary to have a layer of code that handles namespace declarations up front so that the serializer isn't responsible for guaranteeing well-formedness.
And that doesn't even cover extension namespace issues, xsi:types, and so forth, where c14n itself has problems. My project has an entire library layer around DOM and xmlsec to address that stuff.
But you have to be *very* certain about no whitespace changes, above all else. That's the one thing we know will break it. That can have implications for your parser, of course, since you don't want it collapsing whitespace, and schema normalization will also affect linefeeds (not that you're validating, but it's an example).
-- Scott
Re: Signature breaks during I/O and parsing, but cannot resolve with canonicalisation
Posted by ra...@ralphholz.de.
> > The only difference that I can see are the missing namespaces.
>
> That doesn't seem right unless our XML was already indented the same way as
> the transform would.
It's the identity transform - that should not do anything except store the
DOMSource in a Result.
> > No, the problem also occurs if I drop that line (and every other output
> > property). Or do you mean the whole transformer approach will not work?
>
> I have no idea. I certainly believe that's what's breaking it, that or your
> c14n afterwards, which isn't guaranteed to work either. You really have
> little freedom with the XML, almost anything you do could break the
> signature.
Isn't the whole idea of XML Signature and Canonicalisation that exactly this
should not happen? When I hand XML over to some HTTP agent, there are almost
certainly small changes (linefeeds etc.) that happen on the way to the
destination.
> No. Once it's trashed, it's probably trashed.
>
> All you can do is meticulously compare node sets during signing and
> verifying, and see why they're different. Either that or have a really good
> grasp of c14n and actually eyeball it.
Not a solution for me, I fear. The reason I store it to disk and read it back
in again is that I cannot transfer a Document object between VMs by
serialisation -> see my message "Decryption fails on receiving host, but not
on local - pointers?" for this.
In essence, what I need is to transfer the XML between sender and receiver.
What kind of serialisation would you recommend for the transfer?
Ralph
--
For contact details, please see www.ralphholz.de.
RE: Signature breaks during I/O and parsing, but cannot resolve with canonicalisation
Posted by Scott Cantor <ca...@osu.edu>.
> The only difference that I can see are the missing namespaces.
That doesn't seem right unless our XML was already indented the same way as the transform would.
> No, the problem also occurs if I drop that line (and every other output
> property). Or do you mean the whole transformer approach will not work?
I have no idea. I certainly believe that's what's breaking it, that or your c14n afterwards, which isn't guaranteed to work either. You really have little freedom with the XML, almost anything you do could break the signature.
> In any case, if I use a parser to read the file back in and apply a C14N
> transform, the signature should be valid again?
No. Once it's trashed, it's probably trashed.
All you can do is meticulously compare node sets during signing and verifying, and see why they're different. Either that or have a really good grasp of c14n and actually eyeball it.
-- Scott
Re: Signature breaks during I/O and parsing, but cannot resolve with canonicalisation
Posted by ra...@ralphholz.de.
Hi,
> > Looking at my document with kdiff3, I can see that only one line is
> > different:
>
> And there are no whitespace differences?
The only difference that I can see are the missing namespaces.
> > <write out>
> > File file = new File(filename);
> > FileOutputStream f = new FileOutputStream(file);
> > TransformerFactory factory = TransformerFactory.newInstance();
> > Transformer transformer = factory.newTransformer();
> > transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION,
> > "yes");
> > transformer.setOutputProperty("indent", "yes");
>
> Ummm....I'd say that's your problem right there. You can't pretty print
> signed XML, not unless you print it first, parse it back in, then sign it.
No, the problem also occurs if I drop that line (and every other output
property). Or do you mean the whole transformer approach will not work?
In any case, if I use a parser to read the file back in and apply a C14N
transform, the signature should be valid again?
Ralph
--
For contact details, please see www.ralphholz.de.
RE: Signature breaks during I/O and parsing, but cannot resolve with canonicalisation
Posted by Scott Cantor <ca...@osu.edu>.
> Looking at my document with kdiff3, I can see that only one line is
> different:
And there are no whitespace differences?
> The code for write out/read in is the following, although it should not
> matter:
>
> <write out>
> File file = new File(filename);
> FileOutputStream f = new FileOutputStream(file);
> TransformerFactory factory = TransformerFactory.newInstance();
> Transformer transformer = factory.newTransformer();
> transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION,
> "yes");
> transformer.setOutputProperty("indent", "yes");
Ummm....I'd say that's your problem right there. You can't pretty print signed XML, not unless you print it first, parse it back in, then sign it.
-- Scott