You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@santuario.apache.org by Arnaud <ar...@hotmail.com> on 2008/09/25 14:36:59 UTC
Undeclared namespace prefix "ds" error
Hi all,
Since using Chinese in soap messages which had their header encrypt was not
working (and after visiting the XML security page) I upgraded the libraries to
1.4.1 as recommended.
And now I have the following exception on the serverside:
com.ctc.wstx.exc.WstxParsingException: Undeclared namespace prefix "ds"
at [row,col {unknown-source}]: [6,12]
The "[row,col {unknown-source}]: [6,12]" refers to the <ds:KeyInfo> of the soap
header. If I compare the message sent by the client before the upgrade:
<ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
...
</ds:KeyInfo>
and after:
<ds:KeyInfo>
...
</ds:KeyInfo>
I can see the namespace disappeared but I have no idea why and how to fix
that...
If anyone has an idea please let me know because I don't know what to do about
it and I've alread spend many hours on this...
Thanks in advance for any help.
Arnaud.
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
Scott Cantor schrieb:
>> Thus the printout "before c14n" is the doc tree just before c14n. IMHO the
>> Transformer does not add/modify during transformation of the doc tree to a
>> string.
>
> Unless we can prove that, my bet might be that the DOM before that step
> doesn't contain the declaration, and the DOM after it does. Thus the
> transformer might be adding it to "fix-up" the DOM.
>
I don't know what the transfomer does (never looked at the code). Of course I'll
insert some code to check if the xmlns:ds is contained in KeyInfo - I'll keep
you informed :-)
Regards,
Werner
> Can you inject some DOM code to just check the KeyInfo element and see if it
> has xmlns:ds declared? If it does, I think there has to be bug in the c14n
> code, although it sure seems like a blatant one...
>
> -- Scott
>
>
>
>
RE: Undeclared namespace prefix
Posted by Scott Cantor <ca...@osu.edu>.
> Thus the printout "before c14n" is the doc tree just before c14n. IMHO the
> Transformer does not add/modify during transformation of the doc tree to a
> string.
Unless we can prove that, my bet might be that the DOM before that step
doesn't contain the declaration, and the DOM after it does. Thus the
transformer might be adding it to "fix-up" the DOM.
Can you inject some DOM code to just check the KeyInfo element and see if it
has xmlns:ds declared? If it does, I think there has to be bug in the c14n
code, although it sure seems like a blatant one...
-- Scott
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
All,
I've checked the input and output of c14n of xmlsec-1.4.2 and it seems
the c14n does not emit the ds: name space binding of the KeyInfo node.
Attached is the text file of the test run.
The same test but using xmlsec-1.4.1 shows the name space binding before
and after c14n.
The following the code snippet was used to produce the test output file:
...
Canonicalizer c14n =
Canonicalizer.getInstance(Canonicalizer.ALGO_ID_C14N_WITH_COMMENTS);
System.out.println("Before c14n");
System.out.println(XMLUtils.PrettyDocumentToString(doc));
byte[] canonicalMessage = c14n.canonicalizeSubtree(doc);
System.out.println("After c14n: " + new String(canonicalMessage));
...
the XMLUtils.PrettyDocumentToString(doc) (XMLUtils is not the xmlsec XMLUtils
but an own one :-) ) performs as follows:
public class XMLUtils {
public static String PrettyDocumentToString(Document doc) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ElementToStream(doc.getDocumentElement(), baos);
return new String(baos.toByteArray());
}
public static void ElementToStream(Element element, OutputStream out) {
try {
DOMSource source = new DOMSource(element);
StreamResult result = new StreamResult(out);
TransformerFactory transFactory = TransformerFactory.newInstance();
Transformer transformer = transFactory.newTransformer();
transformer.transform(source, result);
} catch (Exception e) {
e.printStackTrace();
}
}
....
Thus the printout "before c14n" is the doc tree just before c14n. IMHO the Transformer
does not add/modify during transformation of the doc tree to a string.
Regards,
Werner
Scott Cantor schrieb:
>> Questions here: does the XML doc that goes into C14N misses any
>> xmlns: declarations at some important positions? If so - where should
>> we include these?
>
> With either incl or excl, the ds namespace prefix should be emitted in both
> spots, since it's visibly used in that element, and not used anywhere up
> above it. So the first/only place it should appear is in the KeyInfo
> element.
>
> If you're saying that the XML listed first is directly passed (in DOM form)
> into the c14n step, and the output is missing the ds namespace, then it's a
> c14n bug.
>
> Otherwise the bug is in the process being used to turn the original XML into
> a DOM that you give to the c14n code.
>
> -- Scott
>
>
>
RE: Undeclared namespace prefix
Posted by Scott Cantor <ca...@osu.edu>.
> Questions here: does the XML doc that goes into C14N misses any
> xmlns: declarations at some important positions? If so - where should
> we include these?
With either incl or excl, the ds namespace prefix should be emitted in both
spots, since it's visibly used in that element, and not used anywhere up
above it. So the first/only place it should appear is in the KeyInfo
element.
If you're saying that the XML listed first is directly passed (in DOM form)
into the c14n step, and the output is missing the ds namespace, then it's a
c14n bug.
Otherwise the bug is in the process being used to turn the original XML into
a DOM that you give to the c14n code.
-- Scott
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
After doing some more tests I came to the following results (no
solution yet). The processing for the test case is as follows:
Create a XML DOM with encrypted data, relevant elements etc using
xmlsec library. This works without any error messages
In the test case a printout of the produced XML document is done,
the a canonicalization of the message that serializes it into
readable XML output. That serialized message is then pares again.
The same test case and flow is used when testing xmlsec 1.4.1 and
1.4.2. The difference between these two runs is after the C14N
step: using xmlsec 1.4.1 and its C14N methods the name space declaration
for ds:KeyInfo is available, using xmlsec 1.4.2 C14N removes these
name space declarations
The attached files show this differences. Both files contain the
pretty print of the XML doc after xmlsec encryption processing but
before C14N, after that the raw output after C14N is shown. As you
can see in both cases the input to C14N is the same (except for the encrypted
data because the key is a random key), and in both cases the ds:KeyInfo
nodes contain the xmlns: declaration.
Only after xmlsec 1.4.2 C14N the ds:KeyInfo node misses the xmlns: declaration.
After xmlsec1.4.1 C14N this is still available.
Questions here: does the XML doc that goes into C14N misses any
xmlns: declarations at some important positions? If so - where should
we include these?
(The pretty print method uses a normal Transformer to transform a DOMSource
to a StreamResult, no other specific step taken here)
Regards,
Werner
Sean Mullan schrieb:
> I don't know what the cause of this regression could be.
>
> The best thing to do is for Arnaud or Peter to file a new bug at
> http://issues.apache.org/bugzilla under the Security project and if
> possible, attach a standalone (i.e. not dependent on WSS4J) test case
> that reproduces the problem.
>
> Thanks,
> Sean
>
<SNIP ----- SNAP>
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
Raul Benito schrieb:
> I was talkinb about the use of them alone and then need to be c14n by
> itself. Anyway I see the point, and I think is one of the sane ones to be
> use outside of the signature. But please in order to not repeat it can you
> send me the junit test case. It will be make the change faster, and it also
> will allowed us not do the same mistake again.
>
Sure - I need to extract a "pure" xmlsec unit test out out the overall WSS4J
unit test.
Just another thought in this context:
The XML Signature specification (and the many other XML specifications in
general) do not restrict usage of all the XML elements they define. Usually
there is no definition of "this is an internal element" or "this is an external
element" (xmlsec implements elements as objects).
For example KeyInfo is used in XML Signature as well as in XML Encryption
specifications. Other elements specified in XML Signature may be re-used
elsewhere (see to the large set of OASIS Web Service specifications :-) ).
In my understanding an implementation of XML Security specification (such as
xmlsec) shall expect that _every_ element could be used in some other context,
even stand-alone. There is no reason why an application shall not be able to
re-use for example a "Reference" element, or a "X509Data" element as a
stand-alone element if the application's XML structure requires this. And of
course an application shall be able to use xmlsec in this case - because it
exists, is tested, and implements these elements.
There is also no such definition as "a sane use outside as signature" - any
application decides on its own what is "sane" or "insane" with respect to the
XML structures it uses.
As a summary: there is *no reason* (and in large parts it's counter-productive)
to single out elements that are defined in the specifications and make them
usable in one specific context only.
Regards,
Werner
>
> On Fri, Oct 3, 2008 at 5:15 PM, Werner Dittmann <Werner.Dittmann@t-online.de
>> wrote:
>
>> Raul Benito schrieb:
>>
>>> Hello,
>>> I think I made the change so I will try to defend it, first of all the use
>>> of KeyInfo out of a Signature it is not a use case I was looking to.
>>>
>> Raul,
>>
>> KeyInfo as such (as an XML element) is not used inside Signature only. If
>> you
>> have a look into the OASIS WSS specification you will see that KeyInfo is
>> used everywhere (nearly everywhere) a key is used, thus also to store
>> information
>> and references to encryption keys and so on. And these are exactly the test
>> cases
>> that break when we use KeyInfo to implement OASIS WSS.
>>
>> Regards,
>> Werner
>>
>>
>> So
>>
>>> perhaps we break it as we don't look at it. And sadly the old api is full
>>> of
>>> internal objects that can be use external. And I see KeyInfo like that.
>>> So in order to fix, can you write a test case that fails and submit a bug,
>>> I
>>> will update the code in SVN head.
>>> Thanks,
>>> Raul
>>>
>>>
>
Re: Undeclared namespace prefix
Posted by Raul Benito <ra...@apache.org>.
I was talkinb about the use of them alone and then need to be c14n by
itself. Anyway I see the point, and I think is one of the sane ones to be
use outside of the signature. But please in order to not repeat it can you
send me the junit test case. It will be make the change faster, and it also
will allowed us not do the same mistake again.
On Fri, Oct 3, 2008 at 5:15 PM, Werner Dittmann <Werner.Dittmann@t-online.de
> wrote:
> Raul Benito schrieb:
>
>> Hello,
>> I think I made the change so I will try to defend it, first of all the use
>> of KeyInfo out of a Signature it is not a use case I was looking to.
>>
>
> Raul,
>
> KeyInfo as such (as an XML element) is not used inside Signature only. If
> you
> have a look into the OASIS WSS specification you will see that KeyInfo is
> used everywhere (nearly everywhere) a key is used, thus also to store
> information
> and references to encryption keys and so on. And these are exactly the test
> cases
> that break when we use KeyInfo to implement OASIS WSS.
>
> Regards,
> Werner
>
>
> So
>
>> perhaps we break it as we don't look at it. And sadly the old api is full
>> of
>> internal objects that can be use external. And I see KeyInfo like that.
>> So in order to fix, can you write a test case that fails and submit a bug,
>> I
>> will update the code in SVN head.
>> Thanks,
>> Raul
>>
>>
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
Raul Benito schrieb:
> Hello,
> I think I made the change so I will try to defend it, first of all the use
> of KeyInfo out of a Signature it is not a use case I was looking to.
Raul,
KeyInfo as such (as an XML element) is not used inside Signature only. If you
have a look into the OASIS WSS specification you will see that KeyInfo is
used everywhere (nearly everywhere) a key is used, thus also to store information
and references to encryption keys and so on. And these are exactly the test cases
that break when we use KeyInfo to implement OASIS WSS.
Regards,
Werner
So
> perhaps we break it as we don't look at it. And sadly the old api is full of
> internal objects that can be use external. And I see KeyInfo like that.
> So in order to fix, can you write a test case that fails and submit a bug, I
> will update the code in SVN head.
> Thanks,
> Raul
>
Re: Undeclared namespace prefix
Posted by Raul Benito <ra...@apache.org>.
Hello,
I think I made the change so I will try to defend it, first of all the use
of KeyInfo out of a Signature it is not a use case I was looking to. So
perhaps we break it as we don't look at it. And sadly the old api is full of
internal objects that can be use external. And I see KeyInfo like that.
So in order to fix, can you write a test case that fails and submit a bug, I
will update the code in SVN head.
Thanks,
Raul
On Fri, Oct 3, 2008 at 4:31 PM, Werner Dittmann <Werner.Dittmann@t-online.de
> wrote:
> Scott Cantor schrieb:
>
>>
>> Well, the bug is holistic. What I mean, and this is what I meant in my
>> original comment, is that the bug is a consequence of the inconsistent XML
>> handling across the range of libraries. It could be viewed as a bug in
>> lots
>> of places.
>>
>> In this specific case, I think it's probably easy to argue that the xmlsec
>> library has no reason not to declare that namespace, I suppose, but
>> ultimately
>> what xmlsec needs to do is define the contract...what is it prepared to
>> promise about the DOM it gives you?
>>
>>
> Well looking at the documentation of the KeyInfo class it says that it will
> look for "ds:KeyInfo" when constructed from an element. This is missing
> when
> "creating from scratch" but one can assume the KeyInfo is somewhat
> symmetric
> in its usage, thus also takes care of the ds: and namespace handling. IMHO,
> if an element creation class has do deal with specific and well defined
> namespaces
> that it should be the task of this class to take care of this. KeyInfo
> class
> creates an element with a namespace prefix, thus it should also add the
> necessary
> namespace attribute to it.
>
> In WSS4J we do it similarly: we return a full Doc tree where elements have
> their associated namespaces attributes thus the application does not need
> to
> know about the specific/efined namespaces.
>
> At least it seems that this modification in xmlsec-1.4.2 has some impacts
> on other
> applications as well because this modification broke an "assumed" contract.
>
> Regards,
> Werner
>
>
> You have to work from that contract, or (as in the case of OpenSAML) build
>> your own XML processing model that takes the responsibility on.
>>
>> But it's also too easy to say "the caller has to fix up namespaces",
>> because
>> unfortunately those namespaces are intrinsic to the signature process. You
>> can't just add them later and expect it to work.
>>
>> -- Scott
>>
>>
>>
>>
>
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
Scott Cantor schrieb:
>
> Well, the bug is holistic. What I mean, and this is what I meant in my
> original comment, is that the bug is a consequence of the inconsistent XML
> handling across the range of libraries. It could be viewed as a bug in lots
> of places.
>
> In this specific case, I think it's probably easy to argue that the xmlsec
> library has no reason not to declare that namespace, I suppose, but
> ultimately
> what xmlsec needs to do is define the contract...what is it prepared to
> promise about the DOM it gives you?
>
Well looking at the documentation of the KeyInfo class it says that it will
look for "ds:KeyInfo" when constructed from an element. This is missing when
"creating from scratch" but one can assume the KeyInfo is somewhat symmetric
in its usage, thus also takes care of the ds: and namespace handling. IMHO,
if an element creation class has do deal with specific and well defined namespaces
that it should be the task of this class to take care of this. KeyInfo class
creates an element with a namespace prefix, thus it should also add the necessary
namespace attribute to it.
In WSS4J we do it similarly: we return a full Doc tree where elements have
their associated namespaces attributes thus the application does not need to
know about the specific/efined namespaces.
At least it seems that this modification in xmlsec-1.4.2 has some impacts on other
applications as well because this modification broke an "assumed" contract.
Regards,
Werner
> You have to work from that contract, or (as in the case of OpenSAML) build
> your own XML processing model that takes the responsibility on.
>
> But it's also too easy to say "the caller has to fix up namespaces", because
> unfortunately those namespaces are intrinsic to the signature process. You
> can't just add them later and expect it to work.
>
> -- Scott
>
>
>
RE: Undeclared namespace prefix
Posted by Scott Cantor <ca...@osu.edu>.
> thanks for your hints :-) . Of course - the xmlns:ds attribute is the
> important part. After checking the attributes of KeyInfo it turned out
> that no xmlns:ds attribute was found.
I figured.
> This functions differs between 1.4.1 and 1.4.2 in some important lines
when
> it comes to add the xmlns:ds attribute to the KeyInfo element (see code
> snippets below). Why was this modification done? I couldn't
finddocumentation
> about this on the xmlsec Web pages. Does KeyInfo expect the user of
KeyInfo
> to set the xmlns:ds attribute? Or is it just a simple, plain old bug? :-)
Well, the bug is holistic. What I mean, and this is what I meant in my
original comment, is that the bug is a consequence of the inconsistent XML
handling across the range of libraries. It could be viewed as a bug in lots
of places.
In this specific case, I think it's probably easy to argue that the xmlsec
library has no reason not to declare that namespace, I suppose, but
ultimately
what xmlsec needs to do is define the contract...what is it prepared to
promise about the DOM it gives you?
You have to work from that contract, or (as in the case of OpenSAML) build
your own XML processing model that takes the responsibility on.
But it's also too easy to say "the caller has to fix up namespaces", because
unfortunately those namespaces are intrinsic to the signature process. You
can't just add them later and expect it to work.
-- Scott
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
Scott,
>
> Sorry, that's not the point. Is there a namespace *declaration* there? The
> fact that the DOM knows that it has that namespace is irrelevant to the c14n
> process, as far as I'm aware. If the DOM doesn't have the xmlns:ds
> attribute, you will not get it back out.
>
thanks for your hints :-) . Of course - the xmlns:ds attribute is the
important part. After checking the attributes of KeyInfo it turned out
that no xmlns:ds attribute was found.
The source code that creates the document uses the
org.apache.xml.security.keys.KeyInfo java class to create the keyinfo element
and to link it into the security header. Digging a bit deeper (via
"org.apache.xml.security.utils.SignatureElementProxy") I found the function
"createElementInSignatureSpace()" in "org.apache.xml.security.utils.XMLUtils".
This functions differs between 1.4.1 and 1.4.2 in some important lines when
it comes to add the xmlns:ds attribute to the KeyInfo element (see code
snippets below). Why was this modification done? I couldn't find documentation
about this on the xmlsec Web pages. Does KeyInfo expect the user of KeyInfo
to set the xmlns:ds attribute? Or is it just a simple, plain old bug? :-)
Regards,
Werner
Code snippets:
**** 1.4.1 :
public static Element createElementInSignatureSpace(Document doc,
String elementName) {
if (doc == null) {
throw new RuntimeException("Document is null");
}
if ((dsPrefix == null) || (dsPrefix.length() == 0)) {
Element element = doc.createElementNS(Constants.SignatureSpecNS,
elementName);
element.setAttributeNS(Constants.NamespaceSpecNS, "xmlns",
Constants.SignatureSpecNS);
return element;
}
String namePrefix=(String) namePrefixes.get(elementName);
if (namePrefix==null) {
StringBuffer tag=new StringBuffer(dsPrefix);
tag.append(':');
tag.append(elementName);
namePrefix=tag.toString();
namePrefixes.put(elementName,namePrefix);
}
Element element = doc.createElementNS(Constants.SignatureSpecNS, namePrefix);
element.setAttributeNS(Constants.NamespaceSpecNS, xmlnsDsPrefix,
Constants.SignatureSpecNS);
return element;
}
**** 1.4.2 :
public static Element createElementInSignatureSpace(Document doc,
String elementName) {
if (doc == null) {
throw new RuntimeException("Document is null");
}
if ((dsPrefix == null) || (dsPrefix.length() == 0)) {
return doc.createElementNS(Constants.SignatureSpecNS, elementName);
}
String namePrefix=(String) namePrefixes.get(elementName);
if (namePrefix==null) {
StringBuffer tag=new StringBuffer(dsPrefix);
tag.append(':');
tag.append(elementName);
namePrefix=tag.toString();
namePrefixes.put(elementName,namePrefix);
}
return doc.createElementNS(Constants.SignatureSpecNS, namePrefix);
}
RE: Undeclared namespace prefix
Posted by Scott Cantor <ca...@osu.edu>.
> the findElement(...) function finds a node, then the code prints its local
> name and the namespace URI:
>
> Before c14n
> KeyInfo, http://www.w3.org/2000/09/xmldsig#
>
> After running c14n the xmlns: namespace binding is gone.
Sorry, that's not the point. Is there a namespace *declaration* there? The
fact that the DOM knows that it has that namespace is irrelevant to the c14n
process, as far as I'm aware. If the DOM doesn't have the xmlns:ds
attribute, you will not get it back out.
> findElemen(...) is just a lookup function to find an element according to
> the parameters and it does not modify the doc tree in any way.
Right, and it also doesn't pay any attention to namespace declarations.
-- Scott
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
After a small update of the test like this:
....
Canonicalizer c14n =
Canonicalizer.getInstance(Canonicalizer.ALGO_ID_C14N_WITH_COMMENTS);
System.out.println("Before c14n");
Node n = WSSecurityUtil.findElement(doc.getDocumentElement(), "KeyInfo", "http://www.w3.org/2000/09/xmldsig#");
if (n == null) {
System.out.println("KeyInfo not fond");
}
else {
System.out.println(n.getLocalName() + ", " + n.getNamespaceURI());
}
System.out.println(XMLUtils.PrettyDocumentToString(doc));
byte[] canonicalMessage = c14n.canonicalizeSubtree(doc);
System.out.println("After c14n: " + new String(canonicalMessage));
....
the findElement(...) function finds a node, then the code prints its local
name and the namespace URI:
Before c14n
KeyInfo, http://www.w3.org/2000/09/xmldsig#
After running c14n the xmlns: namespace binding is gone.
findElemen(...) is just a lookup function to find an element according to
the parameters and it does not modify the doc tree in any way.
Except for the new printout the output of this test of xmlsec-1.4.2 is the
same as attached to my last e-mail
Regards,
Werner
Scott Cantor schrieb:
>> Is is still necessary to file a new bug for this, or has Werner's work
>> covered this?
>
> I think it's still being investigated. If there's no bug filed yet, I would
> suggest doing so, and send Werner the bug number so he can attach the
> results of his tests.
>
> -- Scott
>
>
>
RE: Undeclared namespace prefix
Posted by Scott Cantor <ca...@osu.edu>.
> Is is still necessary to file a new bug for this, or has Werner's work
> covered this?
I think it's still being investigated. If there's no bug filed yet, I would
suggest doing so, and send Werner the bug number so he can attach the
results of his tests.
-- Scott
Re: Undeclared namespace prefix
Posted by "Peter B. West" <li...@pbw.id.au>.
Sean,
Is is still necessary to file a new bug for this, or has Werner's work
covered this?
Peter
sean.mullan wrote:
>
> I don't know what the cause of this regression could be.
>
> The best thing to do is for Arnaud or Peter to file a new bug at
> http://issues.apache.org/bugzilla under the Security project and if
> possible,
> attach a standalone (i.e. not dependent on WSS4J) test case that
> reproduces the
> problem.
>
> Thanks,
> Sean
>
> Werner Dittmann wrote:
>> All,
>>
>> why did it work with 1.4.0 and 1.4.1 then? Were these two version
>> buggy in that case? Or what was changed in 1.4.2 to get this
>> incompatible behaviour?
>>
>> Because we use it in WSS4J we always rely on the backward compatibility.
>> Changing the WSS4J code just to circumvent a problem would cause a major
>> effort to many applications that actually use WSS4J. Before changing
>> WSS4J we shall check if a fix in xmlsec is more appropriate.
>>
>> Regards,
>> Werner
>>
>> Arnaud schrieb:
>>> Peter B. West <lists <at> pbw.id.au> writes:
>>>
>>>> Arnaud,
>>>>
>>>> I'm using 1.4.2, and the problem I had may not be the same as yours,
>>>> but
>>>> see my messages "Unbound prefix after decryption".
>>>>
>>>> In my case, the problem was that the DOM being passed for encryption id
>>>> not have the xmlns attribute set before it was processed. A
>>>> normalizeDocument() in the right place fixed the problem.
>>>>
>>>> Peter
>>>>
>>>> Arnaud wrote:
>>>>> Hi all,
>>>>>
>>>>> Since using Chinese in soap messages which had their header encrypt
>>>>> was not working (and after visiting the XML security page) I
>>>>> upgraded the libraries
>>> to
>>>>> 1.4.1 as recommended.
>>>>> And now I have the following exception on the serverside:
>>>>> com.ctc.wstx.exc.WstxParsingException: Undeclared namespace prefix
>>>>> "ds"
>>>>> at [row,col {unknown-source}]: [6,12]
>>>>>
>>>>> The "[row,col {unknown-source}]: [6,12]" refers to the <ds:KeyInfo>
>>>>> of the
>>> soap
>>>>> header. If I compare the message sent by the client before the
>>>>> upgrade:
>>>>>
>>>>> <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
>>>>> ...
>>>>> </ds:KeyInfo>
>>>>>
>>>>> and after:
>>>>> <ds:KeyInfo> ...
>>>>> </ds:KeyInfo>
>>>>>
>>>>> I can see the namespace disappeared but I have no idea why and how
>>>>> to fix that...
>>>>> If anyone has an idea please let me know because I don't know what
>>>>> to do
>>> about
>>>>> it and I've alread spend many hours on this...
>>>>>
>>>>> Thanks in advance for any help.
>>>>>
>>>>> Arnaud.
>>>
>>> Hi Peter,
>>>
>>> Thanks a lot for your quick reply!
>>> In my case, I'm not using the xml security libs directly but WSS4J
>>> does that for me. Anyway, after reading your post I decided to
>>> download the WSS4J source... And I added the normalizeDocument() just
>>> after WSS4J calls the xml security to handle encryption and
>>> signature... And you are right this fix the problem!!! I will
>>> probably send a mail to the WSS4J team to let them know about this.
>>>
>>> Again thanks a lot!!!
>>>
>>> Arnaud.
>>>
>>>
>>>
>>
>
>
>
--
View this message in context: http://www.nabble.com/Undeclared-namespace-prefix-%22ds%22-error-tp19668706p19752450.html
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
After doing some more tests I came to the following results (no
solution yet). The processing for the test case is as follows:
Create a XML DOM with encrypted data, relevant elements etc using
xmlsec library. This works without any error messages
In the test case a printout of the produced XML document is done,
the a canonicalization of the message that serializes it into
readable XML output. That serialized message is then pares again.
The same test case and flow is used when testing xmlsec 1.4.1 and
1.4.2. The difference between these two runs is after the C14N
step: using xmlsec 1.4.1 and its C14N methods the name space declaration
for ds:KeyInfo is available, using xmlsec 1.4.2 C14N removes these
name space declarations
The attached files show this differences. Both files contain the
pretty print of the XML doc after xmlsec encryption processing but
before C14N, after that the raw output after C14N is shown. As you
can see in both cases the input to C14N is the same (except for the encrypted
data because the key is a random key), and in both cases the ds:KeyInfo
nodes contain the xmlns: declaration.
Only after xmlsec 1.4.2 C14N the ds:KeyInfo node misses the xmlns: declaration.
After xmlsec1.4.1 C14N this is still available.
Questions here: does the XML doc that goes into C14N misses any
xmlns: declarations at some important positions? If so - where should
we include these?
(The pretty print method uses a normal Transformer to transform a DOMSource
to a StreamResult, no other specific step taken here)
Regards,
Werner
Sean Mullan schrieb:
> I don't know what the cause of this regression could be.
>
> The best thing to do is for Arnaud or Peter to file a new bug at
> http://issues.apache.org/bugzilla under the Security project and if
> possible, attach a standalone (i.e. not dependent on WSS4J) test case
> that reproduces the problem.
>
> Thanks,
> Sean
>
<SNIP ----- SNAP>
Re: Undeclared namespace prefix
Posted by Sean Mullan <Se...@Sun.COM>.
I don't know what the cause of this regression could be.
The best thing to do is for Arnaud or Peter to file a new bug at
http://issues.apache.org/bugzilla under the Security project and if possible,
attach a standalone (i.e. not dependent on WSS4J) test case that reproduces the
problem.
Thanks,
Sean
Werner Dittmann wrote:
> All,
>
> why did it work with 1.4.0 and 1.4.1 then? Were these two version
> buggy in that case? Or what was changed in 1.4.2 to get this
> incompatible behaviour?
>
> Because we use it in WSS4J we always rely on the backward compatibility.
> Changing the WSS4J code just to circumvent a problem would cause a major
> effort to many applications that actually use WSS4J. Before changing
> WSS4J we shall check if a fix in xmlsec is more appropriate.
>
> Regards,
> Werner
>
> Arnaud schrieb:
>> Peter B. West <lists <at> pbw.id.au> writes:
>>
>>> Arnaud,
>>>
>>> I'm using 1.4.2, and the problem I had may not be the same as yours, but
>>> see my messages "Unbound prefix after decryption".
>>>
>>> In my case, the problem was that the DOM being passed for encryption id
>>> not have the xmlns attribute set before it was processed. A
>>> normalizeDocument() in the right place fixed the problem.
>>>
>>> Peter
>>>
>>> Arnaud wrote:
>>>> Hi all,
>>>>
>>>> Since using Chinese in soap messages which had their header encrypt
>>>> was not working (and after visiting the XML security page) I
>>>> upgraded the libraries
>> to
>>>> 1.4.1 as recommended.
>>>> And now I have the following exception on the serverside:
>>>> com.ctc.wstx.exc.WstxParsingException: Undeclared namespace prefix "ds"
>>>> at [row,col {unknown-source}]: [6,12]
>>>>
>>>> The "[row,col {unknown-source}]: [6,12]" refers to the <ds:KeyInfo>
>>>> of the
>> soap
>>>> header. If I compare the message sent by the client before the upgrade:
>>>>
>>>> <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
>>>> ...
>>>> </ds:KeyInfo>
>>>>
>>>> and after:
>>>> <ds:KeyInfo> ...
>>>> </ds:KeyInfo>
>>>>
>>>> I can see the namespace disappeared but I have no idea why and how
>>>> to fix that...
>>>> If anyone has an idea please let me know because I don't know what
>>>> to do
>> about
>>>> it and I've alread spend many hours on this...
>>>>
>>>> Thanks in advance for any help.
>>>>
>>>> Arnaud.
>>
>> Hi Peter,
>>
>> Thanks a lot for your quick reply!
>> In my case, I'm not using the xml security libs directly but WSS4J
>> does that for me. Anyway, after reading your post I decided to
>> download the WSS4J source... And I added the normalizeDocument() just
>> after WSS4J calls the xml security to handle encryption and
>> signature... And you are right this fix the problem!!! I will
>> probably send a mail to the WSS4J team to let them know about this.
>>
>> Again thanks a lot!!!
>>
>> Arnaud.
>>
>>
>>
>
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
Scott Cantor schrieb:
>> Because we use it in WSS4J we always rely on the backward compatibility.
>> Changing the WSS4J code just to circumvent a problem would cause a major
>> effort to many applications that actually use WSS4J. Before changing
>> WSS4J we shall check if a fix in xmlsec is more appropriate.
>
> If the issue is with how the XML is being serialized, that's your
> responsibility, not xmlsec's. Namespace declarations can't be assumed. You
> need to manage the DOM rigorously and with great attention to detail.
>
> The fact that a lot of code in these Apache projects doesn't is exactly why
> we did our own.
>
> -- Scott
>
>
>
Looking at the change notes I don't see a relevant entry that explains which
changes could trigger this problem. As said the test case we use is already
several years old, runs perfectly with xmlsec unto and including version 1.4.1
and produces inter-operable XML documents that contain encrypted stuff (only
XML sec encryption is affected). Only using xmlsec 1.4.2 (leaving the rest
of the environment unchanged) triggers this problem.
Did we have a problem in the test case that somehow didn't show up because
of a specific handling (or even problem) in previous xmlsec versions? Sort
of -1 * -1 = +1 ? If so - what was changed in xmlsec that could cause this
behaviour?
Regards,
Werner
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
Peter B. West schrieb:
<SNIP ---- SNAP>
>
> Werner,
>
> It helped a great deal. I'm embarrassed at my misreading of the API.
>
> I forgot about the xmlns *namespace* itself, possibly because
> createElementNS has no need for its explicit expression.
>
> Aside from the embarrassment though, it remains the case that the
> serializer does not need the assistance of setAttributeNS; it does this
> job itself. Is that unreasonable? What other interpretation of
> createElementNS is possible?
I have no idea what a serializer does - however, the c14n serializer
does not include the xmlns:ds="...." attribute if it is not present
at the element. If you the try to parse the resulting XML doc the
parser complains regarding "namespace prefix not bound" (or something
simliar).
See the discussins I hat with Scott and Raul because of a modification
in xmlsec-1.4.2 where the KeyInfo object (element) was modified to omit
the namespace declaration for the ds: prefix. Which resulted in problems.
Regards,
Werner
>
> Peter
>
>
RE: Undeclared namespace prefix
Posted by Scott Cantor <ca...@osu.edu>.
> Granted that it's insane (and I appreciate your opinion about this) is it
> *documented* insanity, or is it defined by implementation? From my limited
> reading, it seems to be a very grey area. The c14n documentation doesn't
> mention the DOM at all, but there may be some documentation on the DOM
side
> that I'm unaware of.
It's not documented any *other* way, that's kind of the only way to say it.
If you follow the text, you find that when handing a c14n engine a DOM
element (and an implied node set of everything below it), nothing in the
spec says to create a namespace declaration if one isn't already present.
The lone exception is the InclusivePrefixes list.
So if it's not declared, it won't get declared, and the XML that results
won't be well-formed (and therefore probably isn't what you meant to sign).
Serializers are not the same as a c14n engine, and it isn't typical to use
one before you sign a DOM, so using serializers to "fix up" namespaces is
really not part of a robust signing application.
-- Scott
RE: Undeclared namespace prefix
Posted by "Peter B. West" <li...@pbw.id.au>.
Scott Cantor wrote:
>
>> Aside from the embarrassment though, it remains the case that the
>> serializer does not need the assistance of setAttributeNS; it does this
>> job itself.
>
> There's no such thing as "the serializer". There are hundreds of things
> called "serializers". DOM3 includes an LS interface that includes "a
> serializer", but that's just one example.
>
> Every serializer has its own properties and behaviors, usually close to
> undocumented, so relying on that isn't robust.
>
> Furthermore, signing takes place in many cases over a DOM, not a
> serialized document. It is not tenable to create the DOM, serialize it,
> parse it back in, and then sign it. So you have to ensure namespace
> correctness in the DOM itself using your own code (or a toolkit designed
> to address that).
>
> The point is, you can't punt this in the manner you described. You may
> find that insane, but I find that most people conclude that about XML
> pretty quickly.
>
> -- Scott
>
>
Granted that it's insane (and I appreciate your opinion about this) is it
*documented* insanity, or is it defined by implementation? From my limited
reading, it seems to be a very grey area. The c14n documentation doesn't
mention the DOM at all, but there may be some documentation on the DOM side
that I'm unaware of.
--
View this message in context: http://www.nabble.com/Undeclared-namespace-prefix-%22ds%22-error-tp19668706p19830062.html
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.
RE: Undeclared namespace prefix
Posted by Scott Cantor <ca...@osu.edu>.
> Aside from the embarrassment though, it remains the case that the
> serializer does not need the assistance of setAttributeNS; it does this
> job itself.
There's no such thing as "the serializer". There are hundreds of things called "serializers". DOM3 includes an LS interface that includes "a serializer", but that's just one example.
Every serializer has its own properties and behaviors, usually close to undocumented, so relying on that isn't robust.
Furthermore, signing takes place in many cases over a DOM, not a serialized document. It is not tenable to create the DOM, serialize it, parse it back in, and then sign it. So you have to ensure namespace correctness in the DOM itself using your own code (or a toolkit designed to address that).
The point is, you can't punt this in the manner you described. You may find that insane, but I find that most people conclude that about XML pretty quickly.
-- Scott
Re: Undeclared namespace prefix
Posted by "Peter B. West" <li...@pbw.id.au>.
Werner Dittmann wrote:
> Peter B. West schrieb:
> <SNIP ----- SNAP>
>>> you're using namespaces.
>>>
>>
>> setAttributeNS doesn't work for me following createElementNS. I get
>> org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or
>> change an object in a way which is incorrect with regard to namespaces.
>>
>> This makes sense, but means that the only way to get the xmlns attribute
>> expressed is to normalizeDocument(). My feeling is that this reinforces
>> my point about the expected behaviour of the serializer.
>>
>> Does it work for you? Maybe I'm doing something wrong.
>>
>
> Here a code snippet how this set attribute stuff works (at least for me
> :-) ),
> also some explanations why the namespace stuff works that way in DOM.
>
> As an example the following element:
> - a Reference element that is part of some OASIS WSS namespace (not the
> same as
> Reference in Signature)
>
> - the element also contains an attribute in another WSS namespace
>
> Example:
> <wsse:Reference
> xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
>
> wsu:Id="someIdentifier"
> xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
>
> ...
> </wsse:Reference>
>
>
> Create the element first:
>
> Element element = doc.createElementNS(WSConstants.WSSE_NS,
> "wsse:Reference");
>
> this creates a namespace qualified element whose namespace is
> "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
>
> (this is the value of the string constant "WSConstants.WSSE_NS").
>
> Please note that wsse:Reference is the element's qualified name
> (Qname), which
> may consist of "prefix:local name" or "local name" alone (refer to W3C
> namespace
> specification).
>
> Why using the namespace URI when creating the element? Refer to the
> following
> sentences given in W3C's namespace spec:
>
> <quote>
> The Prefix provides the namespace prefix part of the qualified name,
> and MUST be
> associated with a namespace URI reference in a namespace declaration.
>
> Note that the prefix functions only as a placeholder for a namespace name.
> Applications SHOULD use the namespace name, not the prefix, in
> constructing
> names whose scope extends beyond the containing document.
> </quote>
>
> This says: the prefix is a convenience (IMHO meant for "humans", not
> necessarily for
> computers :-) ). The real namespace is attached to the element.
>
> Next step is to bind the prefix with a namespace using a namespace
> declaration:
>
> element.setAttributeNS(WSConstants.XMLNS_NS, "xmlns:wsse",
> WSConstants.WSSE_NS);
>
> The namespace declaration attribute:
>
> - "WSConstants.XMLNS_NS" is the string
> "http://www.w3.org/XML/1998/namespace" which
> is fixed and MUST be given this way. This creates an namespace
> qualified attribute
> whose namespace is "http://www.w3.org/XML/1998/namespace".
>
> - "xmlns:wsse" is the name of the attribute. Only the substring "xmlns:"
> defines this
> attribute as a namespace declaration
>
> - the namespace URI of the element, for example
>
> "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
>
> in case of Web Service Security extensions.
>
> Now the namespace qualified attribute of the above element:
>
> element.setAttributeNS(WSConstants.WSU_NS, wsu:Id", id);
> element.setAttributeNS(WSConstants.XMLNS_NS, "xmlns:wsu",
> WSConstants.WSU_NS);
>
> Hope this helped a bit :-) .
>
> Regards,
> Werner
Werner,
It helped a great deal. I'm embarrassed at my misreading of the API.
I forgot about the xmlns *namespace* itself, possibly because
createElementNS has no need for its explicit expression.
Aside from the embarrassment though, it remains the case that the
serializer does not need the assistance of setAttributeNS; it does this
job itself. Is that unreasonable? What other interpretation of
createElementNS is possible?
Peter
>
>>>> <my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>
>>> I'm pretty sure that's because of your bug, using a DOM1 call to
>>> create that
>>> initial attribute.
>>>
>>>> I'm not sure when this arrangement of the namespace and attribute
>>>> occurs,
>>>> but the ordering may be affected by the order of creation -
>>> createElementNS
>>>> first, followed by the attribute setup. In any case, I don't know
>>>> what the
>>>> canonicalizer would do with this. Neither do I know which xmlns:my
>>>> would
>>> win
>>>> when the stream is read next time.
>>> Neither will because that XML isn't well-formed. It won't parse.
>>>
>>> Try using setAttributeNS and I think you'll have more luck.
>>>
>>> -- Scott
--
Peter B. West <http://cv.pbw.id.au/>
Folio <http://defoe.sourceforge.net/folio/>
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
Peter B. West schrieb:
<SNIP ----- SNAP>
>> you're using namespaces.
>>
>
> setAttributeNS doesn't work for me following createElementNS. I get
> org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or
> change an object in a way which is incorrect with regard to namespaces.
>
> This makes sense, but means that the only way to get the xmlns attribute
> expressed is to normalizeDocument(). My feeling is that this reinforces
> my point about the expected behaviour of the serializer.
>
> Does it work for you? Maybe I'm doing something wrong.
>
Here a code snippet how this set attribute stuff works (at least for me :-) ),
also some explanations why the namespace stuff works that way in DOM.
As an example the following element:
- a Reference element that is part of some OASIS WSS namespace (not the same as
Reference in Signature)
- the element also contains an attribute in another WSS namespace
Example:
<wsse:Reference xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
wsu:Id="someIdentifier" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
...
</wsse:Reference>
Create the element first:
Element element = doc.createElementNS(WSConstants.WSSE_NS, "wsse:Reference");
this creates a namespace qualified element whose namespace is
"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
(this is the value of the string constant "WSConstants.WSSE_NS").
Please note that wsse:Reference is the element's qualified name (Qname), which
may consist of "prefix:local name" or "local name" alone (refer to W3C namespace
specification).
Why using the namespace URI when creating the element? Refer to the following
sentences given in W3C's namespace spec:
<quote>
The Prefix provides the namespace prefix part of the qualified name, and MUST be
associated with a namespace URI reference in a namespace declaration.
Note that the prefix functions only as a placeholder for a namespace name.
Applications SHOULD use the namespace name, not the prefix, in constructing
names whose scope extends beyond the containing document.
</quote>
This says: the prefix is a convenience (IMHO meant for "humans", not necessarily for
computers :-) ). The real namespace is attached to the element.
Next step is to bind the prefix with a namespace using a namespace declaration:
element.setAttributeNS(WSConstants.XMLNS_NS, "xmlns:wsse", WSConstants.WSSE_NS);
The namespace declaration attribute:
- "WSConstants.XMLNS_NS" is the string "http://www.w3.org/XML/1998/namespace" which
is fixed and MUST be given this way. This creates an namespace qualified attribute
whose namespace is "http://www.w3.org/XML/1998/namespace".
- "xmlns:wsse" is the name of the attribute. Only the substring "xmlns:" defines this
attribute as a namespace declaration
- the namespace URI of the element, for example
"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
in case of Web Service Security extensions.
Now the namespace qualified attribute of the above element:
element.setAttributeNS(WSConstants.WSU_NS, wsu:Id", id);
element.setAttributeNS(WSConstants.XMLNS_NS, "xmlns:wsu", WSConstants.WSU_NS);
Hope this helped a bit :-) .
Regards,
Werner
>>> <my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>
>> I'm pretty sure that's because of your bug, using a DOM1 call to create that
>> initial attribute.
>>
>>> I'm not sure when this arrangement of the namespace and attribute occurs,
>>> but the ordering may be affected by the order of creation -
>> createElementNS
>>> first, followed by the attribute setup. In any case, I don't know what the
>>> canonicalizer would do with this. Neither do I know which xmlns:my would
>> win
>>> when the stream is read next time.
>> Neither will because that XML isn't well-formed. It won't parse.
>>
>> Try using setAttributeNS and I think you'll have more luck.
>>
>> -- Scott
>
Re: Undeclared namespace prefix
Posted by "Peter B. West" <li...@pbw.id.au>.
Scott Cantor wrote:
>> I am uncertain about the underlying assumptions here. Please correct me
>> where I am wrong in what follows.
>
> I have to warn you, I'm not a c14n expert, and I know approximately zero
> about XPath. What I'm knowledgable about are the practical aspects of using
> a DOM and signing.
>
And I know nothing about c14n, DOM or signing, and a little about XPath.
>> When I construct a DOM with default configuration using createElementNS,
> the
>> elements have an associated prefix, namespace and local name. I assume
> such
>> a DOM is a valid representation of an XML document, because when I
> serialize
>> such a DOM, I get a valid octet stream.
>
> No, you're incorrect there. First of all, "valid" isn't a term of art there
> unless you apply a DTD or schema. The term is "well-formed", and it is the
> case that unless you add a namespace declaration attribute to the element
> (or it's in scope above), the XML will not be well formed if you simply
> output the DOM directly.
>
Point taken. What I meant was just that the the DOM serializes with the
correct namespaces.
> Now, if you run it through a fix-up in which the serializer "just fixes
> stuff" like adding missing declarations, then yes, it might come out
> well-formed.
>
The serializer associated with JDK 1.6.0 does precisely this. I don't
know whether such behaviour is mandated or discretionary, but it is
entirely reasonable. Without is, what is the point of having a method
createElementNS? It's presence, along with getPrefix, getNamespaceURI
and getLocalName, simply muddies the waters.
And if the serializer can interpret the DOM in such a (to me,
reasonable) way, why can't the canonicalizer?
I can't see that anyone would deliberately rely on the fact that, having
created a namespaced element with createElementNS (including the prefix
in the qualified name), that subsequent canonicalization would _not_
include the namespace attribute. I can, however, see that there might be
code out there that is de facto depending on this behaviour.
> Also, I believe that formally speaking, a DOM itself has no well-formedness
> property. I could be wrong about that. If it did, then I'm saying that in
> your example, the DOM isn't.
>
>> However, such a DOM does not, before
>> I normalizeDocument, have attribute nodes containing namespace
>> specifications. It has something, though, because it serializes correctly.
>
> It serializes because any DOM can be serialized. "Correctness" depends on
> context.
>
>> It seems to me that this situation more closely represents the XPath data
>> model than does the same DOM after normalization, in that namespace nodes
>> cannot be found by searching the attribute axis. The can, however, be
> found
>> by means specific to namespaces: getprefix, getNamespaceURI, getLocalName.
>> After normalizeDocument, these relationships still pertain.
>
> That's true, but the fact is that setting a prefix and namespace URI on an
> element does NOT imply that the prefix is bound to that namespace. That
> might be nice, but that's not how it works in a DOM.
>
>> If, however, I attempt to set the namespace/attribute node specifically,
>> using, say,
>> root.setAttribute("xmlns:my", "a:b:d");
>> things get messy.
>
> Welcoome to XML. And FWIW, you'd use setAttributeNS. NEVER use DOM1 calls if
> you're using namespaces.
>
setAttributeNS doesn't work for me following createElementNS. I get
org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or
change an object in a way which is incorrect with regard to namespaces.
This makes sense, but means that the only way to get the xmlns attribute
expressed is to normalizeDocument(). My feeling is that this reinforces
my point about the expected behaviour of the serializer.
Does it work for you? Maybe I'm doing something wrong.
>> <my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>
>
> I'm pretty sure that's because of your bug, using a DOM1 call to create that
> initial attribute.
>
>> I'm not sure when this arrangement of the namespace and attribute occurs,
>> but the ordering may be affected by the order of creation -
> createElementNS
>> first, followed by the attribute setup. In any case, I don't know what the
>> canonicalizer would do with this. Neither do I know which xmlns:my would
> win
>> when the stream is read next time.
>
> Neither will because that XML isn't well-formed. It won't parse.
>
> Try using setAttributeNS and I think you'll have more luck.
>
> -- Scott
--
Peter B. West <http://cv.pbw.id.au/>
Folio <http://defoe.sourceforge.net/folio/>
RE: Undeclared namespace prefix
Posted by Scott Cantor <ca...@osu.edu>.
> I am uncertain about the underlying assumptions here. Please correct me
> where I am wrong in what follows.
I have to warn you, I'm not a c14n expert, and I know approximately zero
about XPath. What I'm knowledgable about are the practical aspects of using
a DOM and signing.
> When I construct a DOM with default configuration using createElementNS,
the
> elements have an associated prefix, namespace and local name. I assume
such
> a DOM is a valid representation of an XML document, because when I
serialize
> such a DOM, I get a valid octet stream.
No, you're incorrect there. First of all, "valid" isn't a term of art there
unless you apply a DTD or schema. The term is "well-formed", and it is the
case that unless you add a namespace declaration attribute to the element
(or it's in scope above), the XML will not be well formed if you simply
output the DOM directly.
Now, if you run it through a fix-up in which the serializer "just fixes
stuff" like adding missing declarations, then yes, it might come out
well-formed.
Also, I believe that formally speaking, a DOM itself has no well-formedness
property. I could be wrong about that. If it did, then I'm saying that in
your example, the DOM isn't.
> However, such a DOM does not, before
> I normalizeDocument, have attribute nodes containing namespace
> specifications. It has something, though, because it serializes correctly.
It serializes because any DOM can be serialized. "Correctness" depends on
context.
> It seems to me that this situation more closely represents the XPath data
> model than does the same DOM after normalization, in that namespace nodes
> cannot be found by searching the attribute axis. The can, however, be
found
> by means specific to namespaces: getprefix, getNamespaceURI, getLocalName.
> After normalizeDocument, these relationships still pertain.
That's true, but the fact is that setting a prefix and namespace URI on an
element does NOT imply that the prefix is bound to that namespace. That
might be nice, but that's not how it works in a DOM.
> If, however, I attempt to set the namespace/attribute node specifically,
> using, say,
> root.setAttribute("xmlns:my", "a:b:d");
> things get messy.
Welcoome to XML. And FWIW, you'd use setAttributeNS. NEVER use DOM1 calls if
you're using namespaces.
> <my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>
I'm pretty sure that's because of your bug, using a DOM1 call to create that
initial attribute.
> I'm not sure when this arrangement of the namespace and attribute occurs,
> but the ordering may be affected by the order of creation -
createElementNS
> first, followed by the attribute setup. In any case, I don't know what the
> canonicalizer would do with this. Neither do I know which xmlns:my would
win
> when the stream is read next time.
Neither will because that XML isn't well-formed. It won't parse.
Try using setAttributeNS and I think you'll have more luck.
-- Scott
RE: Undeclared namespace prefix
Posted by "Peter B. West" <li...@pbw.id.au>.
Scott Cantor wrote:
>
>> Because we use it in WSS4J we always rely on the backward compatibility.
>> Changing the WSS4J code just to circumvent a problem would cause a major
>> effort to many applications that actually use WSS4J. Before changing
>> WSS4J we shall check if a fix in xmlsec is more appropriate.
>
> If the issue is with how the XML is being serialized, that's your
> responsibility, not xmlsec's. Namespace declarations can't be assumed. You
> need to manage the DOM rigorously and with great attention to detail.
>
> The fact that a lot of code in these Apache projects doesn't is exactly
> why
> we did our own.
>
> -- Scott
>
>
Scott,
I am uncertain about the underlying assumptions here. Please correct me
where I am wrong in what follows.
The problem I have seen involves canonicalization. The c14n processing model
is defined in terms of the XPath data model. As I understand it, the XPath
data model separates the attribute and namespace axes. You can't find the
namespaces on the attribute axis.
Implementations of c14n may be based on XPath, or not. They may accept a
node set as input, or not. When they do accept a node set, whether that node
set is, in fact, a node set returned by XPath or a DOM nodeset, should it
not be treated in a manner which is compatible with the XPath data model?
When I construct a DOM with default configuration using createElementNS, the
elements have an associated prefix, namespace and local name. I assume such
a DOM is a valid representation of an XML document, because when I serialize
such a DOM, I get a valid octet stream. However, such a DOM does not, before
I normalizeDocument, have attribute nodes containing namespace
specifications. It has something, though, because it serializes correctly.
It seems to me that this situation more closely represents the XPath data
model than does the same DOM after normalization, in that namespace nodes
cannot be found by searching the attribute axis. The can, however, be found
by means specific to namespaces: getprefix, getNamespaceURI, getLocalName.
After normalizeDocument, these relationships still pertain.
If, however, I attempt to set the namespace/attribute node specifically,
using, say,
root.setAttribute("xmlns:my", "a:b:d");
things get messy. For example,
Document doc = newDocument();
Element root = (Element) doc.createElementNS("a:b:c", "my:name");
root.setAttribute("xmlns:my", "a:b:b");
doc.appendChild(root);
System.err.println("root ns: "+root.getNamespaceURI());
System.err.println("prefix: "+root.getPrefix());
System.err.println("localname: "+root.getLocalName());
NamedNodeMap map = root.getAttributes();
for ( int i = 0; i < map.getLength(); i++ ) {
Attr attr = (Attr) map.item(i);
System.err.println("attr name: "+attr.getName());
System.err.println("value: "+attr.getValue());
}
serializeTestData(doc, System.err);
System.err.println("");
doc.normalizeDocument();
System.err.println("root ns: "+root.getNamespaceURI());
System.err.println("prefix: "+root.getPrefix());
System.err.println("localname: "+root.getLocalName());
map = root.getAttributes();
for ( int i = 0; i < map.getLength(); i++ ) {
Attr attr = (Attr) map.item(i);
System.err.println("attr name: "+attr.getName());
System.err.println("value: "+attr.getValue());
}
serializeTestData(doc, System.err);
System.err.println("");
gives me:
root ns: a:b:c
prefix: my
localname: name
attr name: xmlns:my
value: a:b:b
<my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>
root ns: a:b:c
prefix: my
localname: name
attr name: xmlns:my
value: a:b:c
attr name: xmlns:my
value: a:b:b
<my:name xmlns:my="a:b:c" xmlns:my="a:b:b"/>
I'm not sure when this arrangement of the namespace and attribute occurs,
but the ordering may be affected by the order of creation - createElementNS
first, followed by the attribute setup. In any case, I don't know what the
canonicalizer would do with this. Neither do I know which xmlns:my would win
when the stream is read next time.
The situation is artificial, but the getPrefix, getNamespaceURI and
getLocalName values are constant.
Peter
--
View this message in context: http://www.nabble.com/Undeclared-namespace-prefix-%22ds%22-error-tp19668706p19773709.html
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.
RE: Undeclared namespace prefix
Posted by Scott Cantor <ca...@osu.edu>.
> Because we use it in WSS4J we always rely on the backward compatibility.
> Changing the WSS4J code just to circumvent a problem would cause a major
> effort to many applications that actually use WSS4J. Before changing
> WSS4J we shall check if a fix in xmlsec is more appropriate.
If the issue is with how the XML is being serialized, that's your
responsibility, not xmlsec's. Namespace declarations can't be assumed. You
need to manage the DOM rigorously and with great attention to detail.
The fact that a lot of code in these Apache projects doesn't is exactly why
we did our own.
-- Scott
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
All,
why did it work with 1.4.0 and 1.4.1 then? Were these two version
buggy in that case? Or what was changed in 1.4.2 to get this
incompatible behaviour?
Because we use it in WSS4J we always rely on the backward compatibility.
Changing the WSS4J code just to circumvent a problem would cause a major
effort to many applications that actually use WSS4J. Before changing
WSS4J we shall check if a fix in xmlsec is more appropriate.
Regards,
Werner
Arnaud schrieb:
> Peter B. West <lists <at> pbw.id.au> writes:
>
>> Arnaud,
>>
>> I'm using 1.4.2, and the problem I had may not be the same as yours, but
>> see my messages "Unbound prefix after decryption".
>>
>> In my case, the problem was that the DOM being passed for encryption id
>> not have the xmlns attribute set before it was processed. A
>> normalizeDocument() in the right place fixed the problem.
>>
>> Peter
>>
>> Arnaud wrote:
>>> Hi all,
>>>
>>> Since using Chinese in soap messages which had their header encrypt was not
>>> working (and after visiting the XML security page) I upgraded the libraries
> to
>>> 1.4.1 as recommended.
>>> And now I have the following exception on the serverside:
>>> com.ctc.wstx.exc.WstxParsingException: Undeclared namespace prefix "ds"
>>> at [row,col {unknown-source}]: [6,12]
>>>
>>> The "[row,col {unknown-source}]: [6,12]" refers to the <ds:KeyInfo> of the
> soap
>>> header. If I compare the message sent by the client before the upgrade:
>>>
>>> <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
>>> ...
>>> </ds:KeyInfo>
>>>
>>> and after:
>>> <ds:KeyInfo>
>>> ...
>>> </ds:KeyInfo>
>>>
>>> I can see the namespace disappeared but I have no idea why and how to fix
>>> that...
>>> If anyone has an idea please let me know because I don't know what to do
> about
>>> it and I've alread spend many hours on this...
>>>
>>> Thanks in advance for any help.
>>>
>>> Arnaud.
>
> Hi Peter,
>
> Thanks a lot for your quick reply!
> In my case, I'm not using the xml security libs directly but WSS4J does that
> for me. Anyway, after reading your post I decided to download the WSS4J
> source... And I added the normalizeDocument() just after WSS4J calls the xml
> security to handle encryption and signature... And you are right this fix the
> problem!!!
> I will probably send a mail to the WSS4J team to let them know about this.
>
> Again thanks a lot!!!
>
> Arnaud.
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: wss4j-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: wss4j-dev-help@ws.apache.org
Re: Undeclared namespace prefix
Posted by Werner Dittmann <We...@t-online.de>.
All,
why did it work with 1.4.0 and 1.4.1 then? Were these two version
buggy in that case? Or what was changed in 1.4.2 to get this
incompatible behaviour?
Because we use it in WSS4J we always rely on the backward compatibility.
Changing the WSS4J code just to circumvent a problem would cause a major
effort to many applications that actually use WSS4J. Before changing
WSS4J we shall check if a fix in xmlsec is more appropriate.
Regards,
Werner
Arnaud schrieb:
> Peter B. West <lists <at> pbw.id.au> writes:
>
>> Arnaud,
>>
>> I'm using 1.4.2, and the problem I had may not be the same as yours, but
>> see my messages "Unbound prefix after decryption".
>>
>> In my case, the problem was that the DOM being passed for encryption id
>> not have the xmlns attribute set before it was processed. A
>> normalizeDocument() in the right place fixed the problem.
>>
>> Peter
>>
>> Arnaud wrote:
>>> Hi all,
>>>
>>> Since using Chinese in soap messages which had their header encrypt was not
>>> working (and after visiting the XML security page) I upgraded the libraries
> to
>>> 1.4.1 as recommended.
>>> And now I have the following exception on the serverside:
>>> com.ctc.wstx.exc.WstxParsingException: Undeclared namespace prefix "ds"
>>> at [row,col {unknown-source}]: [6,12]
>>>
>>> The "[row,col {unknown-source}]: [6,12]" refers to the <ds:KeyInfo> of the
> soap
>>> header. If I compare the message sent by the client before the upgrade:
>>>
>>> <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
>>> ...
>>> </ds:KeyInfo>
>>>
>>> and after:
>>> <ds:KeyInfo>
>>> ...
>>> </ds:KeyInfo>
>>>
>>> I can see the namespace disappeared but I have no idea why and how to fix
>>> that...
>>> If anyone has an idea please let me know because I don't know what to do
> about
>>> it and I've alread spend many hours on this...
>>>
>>> Thanks in advance for any help.
>>>
>>> Arnaud.
>
> Hi Peter,
>
> Thanks a lot for your quick reply!
> In my case, I'm not using the xml security libs directly but WSS4J does that
> for me. Anyway, after reading your post I decided to download the WSS4J
> source... And I added the normalizeDocument() just after WSS4J calls the xml
> security to handle encryption and signature... And you are right this fix the
> problem!!!
> I will probably send a mail to the WSS4J team to let them know about this.
>
> Again thanks a lot!!!
>
> Arnaud.
>
>
>
Re: Undeclared namespace prefix
Posted by Arnaud <ar...@hotmail.com>.
Peter B. West <lists <at> pbw.id.au> writes:
>
> Arnaud,
>
> I'm using 1.4.2, and the problem I had may not be the same as yours, but
> see my messages "Unbound prefix after decryption".
>
> In my case, the problem was that the DOM being passed for encryption id
> not have the xmlns attribute set before it was processed. A
> normalizeDocument() in the right place fixed the problem.
>
> Peter
>
> Arnaud wrote:
> > Hi all,
> >
> > Since using Chinese in soap messages which had their header encrypt was not
> > working (and after visiting the XML security page) I upgraded the libraries
to
> > 1.4.1 as recommended.
> > And now I have the following exception on the serverside:
> > com.ctc.wstx.exc.WstxParsingException: Undeclared namespace prefix "ds"
> > at [row,col {unknown-source}]: [6,12]
> >
> > The "[row,col {unknown-source}]: [6,12]" refers to the <ds:KeyInfo> of the
soap
> > header. If I compare the message sent by the client before the upgrade:
> >
> > <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
> > ...
> > </ds:KeyInfo>
> >
> > and after:
> > <ds:KeyInfo>
> > ...
> > </ds:KeyInfo>
> >
> > I can see the namespace disappeared but I have no idea why and how to fix
> > that...
> > If anyone has an idea please let me know because I don't know what to do
about
> > it and I've alread spend many hours on this...
> >
> > Thanks in advance for any help.
> >
> > Arnaud.
>
Hi Peter,
Thanks a lot for your quick reply!
In my case, I'm not using the xml security libs directly but WSS4J does that
for me. Anyway, after reading your post I decided to download the WSS4J
source... And I added the normalizeDocument() just after WSS4J calls the xml
security to handle encryption and signature... And you are right this fix the
problem!!!
I will probably send a mail to the WSS4J team to let them know about this.
Again thanks a lot!!!
Arnaud.
Re: Undeclared namespace prefix "ds" error
Posted by "Peter B. West" <li...@pbw.id.au>.
Arnaud,
I'm using 1.4.2, and the problem I had may not be the same as yours, but
see my messages "Unbound prefix after decryption".
In my case, the problem was that the DOM being passed for encryption id
not have the xmlns attribute set before it was processed. A
normalizeDocument() in the right place fixed the problem.
Peter
Arnaud wrote:
> Hi all,
>
> Since using Chinese in soap messages which had their header encrypt was not
> working (and after visiting the XML security page) I upgraded the libraries to
> 1.4.1 as recommended.
> And now I have the following exception on the serverside:
> com.ctc.wstx.exc.WstxParsingException: Undeclared namespace prefix "ds"
> at [row,col {unknown-source}]: [6,12]
>
> The "[row,col {unknown-source}]: [6,12]" refers to the <ds:KeyInfo> of the soap
> header. If I compare the message sent by the client before the upgrade:
>
> <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
> ...
> </ds:KeyInfo>
>
> and after:
> <ds:KeyInfo>
> ...
> </ds:KeyInfo>
>
> I can see the namespace disappeared but I have no idea why and how to fix
> that...
> If anyone has an idea please let me know because I don't know what to do about
> it and I've alread spend many hours on this...
>
> Thanks in advance for any help.
>
> Arnaud.
--
Peter B. West <http://cv.pbw.id.au/>
Folio <http://defoe.sourceforge.net/folio/>