You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@santuario.apache.org by Wolfgang Freis <wo...@yale.edu> on 2007/10/30 19:47:15 UTC

Still stuck with problem. Re: Problem decrypting elements

Hello all,

I'm still stuck with this problem and don't know what else to try.  I'm
basically using the sample code with some modifications to handle multiple
elements.

Encryption works fine with child elements or contents both when a single or
multiple elements with the same name are encrypted.  However, decryption
only works when there is a single element or content node to decrypt.  If
there are more, an exception is thrown:

  Oct 30, 2007 2:28:18 PM org.apache.xml.security.encryption.XMLCipher
    decryptToByteArray
  SEVERE: XMLCipher::decryptElement called without a key and unable to
    resolve Exception in thread "main"
    org.apache.xml.security.encryption.XMLEncryptionException: No Key
    Encryption Key loaded and cannot determine using key resolvers

Stepping through the code in my IDE, I can see that the key gets loaded. 
What is missing, though, is the KeyInfo.  Method
decryptToByteArray(Element element) in class XMLCipher tries to get the
KeyInfo:

   KeyInfo ki = encryptedData.getKeyInfo();

When ki gets evaluated it is null, however, and the exception is thrown. 
Again, it only happens when multiple elements are encrypted.

Obviously, I'm missing something here but neither the documentation nor the
email archive have given me any clues on how to solve this.  Any help will
be greatly appreciated.

Thanks,
Wolfgang







Quoting Wolfgang Freis <wo...@yale.edu>:

> Hello,
>
> I'm new to working with XML encryption and have come across a problem.
> Neither the email archive nor have brought me closer to a solution.
>
> Google Starting with the sample files Encrypter.java and Decrypter.java
> I
> have managed to en- and decrypt elements by name.  However, decryption
> works only when there is only one element encrypted.  If I encrypt
> multiple elements by name--as siblings or as children of different
> parents--the decryptor throws an exception:
>
> Oct 26, 2007 3:56:45 PM org.apache.xml.security.encryption.XMLCipher
>     decryptToByteArray
> SEVERE: XMLCipher::decryptElement called without a key and unable to
>     resolve
> Exception in thread "main"
> org.apache.xml.security.encryption.XMLEncryptionException:
>     No Key Encryption Key loaded and cannot determine using key
> resolvers
>
> My IDE tells that the key file was loaded and, as I said, the decryption
> works fine when only one element of the XML is encrypted.  I tried
> decrypting by iterating over the NodeList returned from
> getElementsByTagNameNS(namespaceURI, localName) and trying to just grab
> the first item.  Both generate the same exception.  From what I can see
> in
> the IDE is that the method decryptToByteArray(Element element) called
> from
> doFinal( document, encryptedDataElement) throws the exception, but it
> escapes me why that happens.
>
> Any help will be greatly appreciated.
>
> Thanks,
> Wolfgang
>
>
>

Re: Still stuck with problem. Re: Problem decrypting elements

Posted by Brent Putman <pu...@georgetown.edu>.

rlyders wrote:
>
> Once the element has been encrypted and signed, the result is shown below.
> The web service that we are integrating with is not a full fledged SOAP WSDL
> web service, but rather it simply uses HTTPS and XML. Our existing XML
> document format where the XML Signature is enclosed by the EncryptedData
> element seems to work well with the with the web service we are integrating
> to. 

Well, it may "work", but it's completely violating the data model put 
forth by the XML Encryption spec.  So you/they definitely can't expect 
it to interop with clients or other libraries that respect that 
specification.


> It is only in my local Java unit tests on my development computer that I
> am seeing these problems with the Signature's KeyInfo element getting in the
> way of decryption. The web service is written in .Net while my stuff is in
> Java, so they must be doing something on their side in .Net to allow for a
> Signature element enclosed in the EncryptedData element.
>   


I don't know much about .Net libraries for doing XML Signature and XML 
Encryption - or perhaps the web service side has rolled its own impl of 
those things - but it's definitely taking creative liberties as far as 
those specs go, if it's allowing the Signature child of the 
EncryptedData element.


>
>
> I will check with the developer of the web service to see if we can slightly
> adjust the encryption method so that the EncryptedData and Signature
> elements are siblings contained within the <Orders> element. 
>   

That's one option I suppose.  It would seem more natural and simpler to 
me, however, that you might want to sign the Orders element with an 
enveloped Signature, and then encrypt the whole thing.  Then on the 
receiver reverse the order: decrypt and then verify the signature.

You might also try persuading them to look at the WS-Security 
specification, a large part of which is specifically about doing XML 
Signature and XML Encryption in conjunction with SOAP web services 
messaging.   That makes heavy use of SOAP headers, though, which usually 
implies some kind of more formal SOAP processing engine.


> It would be a great help if you had a URI that I can reference to base a
> change request on that mentions our illegal format having the enveloped
> Signature as a child element of the EncryptedData element? I haven't been
> able to find any information yet that specifically states that it is illegal
> to place the Signature inside the EncryptedData element. It would be nice to
> have it documented if possible.
>   

Oh, absolutely, that's easy.  The XML Encryption data model is defined 
by W3C Schema, and it's very clear about what the xenc:EncryptedData 
element looks like.  The full spec is here:

http://www.w3.org/TR/xmlenc-core/



In particular the schema xsd is here:

http://www.w3.org/TR/xmlenc-core/xenc-schema.xsd



And within that schema, the EncryptedData is defined so:

<element name='EncryptedData' type='xenc:EncryptedDataType'/>
  <complexType name='EncryptedDataType'>
    <complexContent>
      <extension base='xenc:EncryptedType'>
       </extension>
    </complexContent>
  </complexType>


  <complexType name='EncryptedType' abstract='true'>

    <sequence>
      <element name='EncryptionMethod' type='xenc:EncryptionMethodType'
       minOccurs='0'/>
      <element ref='ds:KeyInfo' minOccurs='0'/>
      <element ref='xenc:CipherData'/>
      <element ref='xenc:EncryptionProperties' minOccurs='0'/>
    </sequence>
    <attribute name='Id' type='ID' use='optional'/>
    <attribute name='Type' type='anyURI' use='optional'/>
    <attribute name='MimeType' type='string' use='optional'/>

    <attribute name='Encoding' type='anyURI' use='optional'/>
  </complexType>

> Thanks for all your help and time.
>   


No problem.

--Brent

Re: Still stuck with problem. Re: Problem decrypting elements

Posted by rlyders <Ri...@Lyders.Net>.
Hi Brent,
yes, I do in fact have the Signature inside the EncryptedData element as you
suggested. I'll have to check with the web service that I am integrating
with to see if it will support some other grouping of the XML elements.
Currently the web service specification asks for one root <Orders> element
that contains multiple <Order> elements as shown below:

<?xml version="1.0" encoding="UTF-8"?>
<Orders xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="OrderRequest.xsd">
  <Order>
    <DocumentNumber>24773064</DocumentNumber>
  </Order>
  <Order>
    <DocumentNumber>24773064</DocumentNumber>
  </Order>
</Orders>

Once the element has been encrypted and signed, the result is shown below.
The web service that we are integrating with is not a full fledged SOAP WSDL
web service, but rather it simply uses HTTPS and XML. Our existing XML
document format where the XML Signature is enclosed by the EncryptedData
element seems to work well with the with the web service we are integrating
to. It is only in my local Java unit tests on my development computer that I
am seeing these problems with the Signature's KeyInfo element getting in the
way of decryption. The web service is written in .Net while my stuff is in
Java, so they must be doing something on their side in .Net to allow for a
Signature element enclosed in the EncryptedData element.

Here is what the resulting encrypted data looks like:
<?xml version="1.0" encoding="UTF-8"?>
<xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#"
                    Type="http://www.w3.org/2001/04/xmlenc#Element">
  <ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
  ...
    <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
    ...
    </ds:KeyInfo>
  </ds:Signature>
  <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#..."
                         xmlns:xenc="http://www.w3.org/2001/04/xmlenc#"/>
  <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
  ...
  </ds:KeyInfo>
  <xenc:CipherData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
    <xenc:CipherValue xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
    ...
    </xenc:CipherValue>
  </xenc:CipherData>
</xenc:EncryptedData>

I will check with the developer of the web service to see if we can slightly
adjust the encryption method so that the EncryptedData and Signature
elements are siblings contained within the <Orders> element. 

It would be a great help if you had a URI that I can reference to base a
change request on that mentions our illegal format having the enveloped
Signature as a child element of the EncryptedData element? I haven't been
able to find any information yet that specifically states that it is illegal
to place the Signature inside the EncryptedData element. It would be nice to
have it documented if possible.

Thanks for all your help and time.


Brent Putman wrote:
> 
> Brent Putman wrote:
> 
> So if you had a DOM tree something like:
> 
> <xenc:EncryptedData>
>    <ds:Signature>
>         <ds:KeyInfo>...</ds:KeyInfo>
>    </ds:Signature>
>    <ds:KeyInfo>
>         <xenc:EncryptedKey>...</xenc:EncryptedKey>
>    </ds:KeyInfo>
> </xenc:EncryptedData>
> 
> It actually would find the Signature KeyInfo first (due to document
> order)  and incorrectly set that one as the EncryptedData's KeyInfo
> member.  And that would account for what you described.
> 
> But like I said:  you can't place the Signature there like that inside
> the EncryptedData, it's not legal.
> 

-- 
View this message in context: http://www.nabble.com/Problem-decrypting-elements-tf4699611.html#a14130034
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.

Re: Still stuck with problem. Re: Problem decrypting elements

Posted by Brent Putman <pu...@georgetown.edu>.

Brent Putman wrote:
>
>
> I don't see how it could be getting the two KeyInfo's confused. All the
> Apache EncryptedData.getKeyInfo() does is return a data member, it
> doesn't do any searching or resolution.
>
>   

> ........................
>   

> Sound like you're trying to sign the EncryptedData itself?
>
> <xenc:EncryptedData>
>  <ds:Signature>...</ds:Signature>
> </xenc:EncryptedData>
>
>
> Note that you can't do that.   It's not schema valid, xenc:EncryptedData
> (actually xenc:EncryptedType) doesn't have an open content model, you
> can't just add arbitrary child elements.
>
> And even if it did, I'm still not seeing how the Apache EncryptedData
> would be pulling the KeyInfo out of the ds:Signature rather than it's
> own immediate child.
>   

Just to follow up on my own post:  I actually do see now why that is
happening, if you are doing the above.  The factory code in XMLCipher
that unmarshals the EncryptedData DOM Element into the Apache
EncryptedData object uses Element#getElementsByTagNameNS  (which finds
*all* matching descendants, not just immediate children) and takes the
first one:

Element keyInfoElement =
                (Element) element.getElementsByTagNameNS(
                    Constants.SignatureSpecNS,
Constants._TAG_KEYINFO).item(0);


(Which might be a buggy way to do that in and of itself, since the
EncryptionMethod which order-wise precedes the real KeyInfo has an open
content model and might theoretically somehow have a ds:KeyInfo buried
in there somewhere....  Maybe should be fixed to iterate over the
immediate children only?)


So if you had a DOM tree something like:


<xenc:EncryptedData>

   <ds:Signature>
        <ds:KeyInfo>...</ds:KeyInfo>
   </ds:Signature>

   <ds:KeyInfo>
        <xenc:EncryptedKey>...</xenc:EncryptedKey>
   </ds:KeyInfo>

</xenc:EncryptedData>



It actually would find the Signature KeyInfo first (due to document
order)  and incorrectly set that one as the EncryptedData's KeyInfo
member.  And that would account for what you described.

But like I said:  you can't place the Signature there like that inside
the EncryptedData, it's not legal.

Hope that helps,
Brent

Re: Still stuck with problem. Re: Problem decrypting elements

Posted by Brent Putman <pu...@georgetown.edu>.
Hi,
I'm not completely clear on what you're doing, but it sounds like at a
minimum you're perhaps doing something that's not legal. More below.


rlyders wrote:
> Brent,
> Your comments lead me to debug the XMCipher java class to find out that my
> enveloped Signature is confusing the default decryption. 

Can you clarify what you mean when you say you are using an enveloped
signature?  Enveloped by what?

If you mean you have:

<Foo>
   <ds:Signature>...</ds:Signature>
</Foo>


then when you encrypt that Foo element, the signature (and it's
subordinate KeyInfo) will be completely invisible within the resultant
xenc:EncryptedData (prior to and during decryption), because it's part
of the encrypted data (encoded octets within xenc:CipherValue).  No way
it could be finding that signature KeyInfo there.

If you mean you have:

<Foo>
  <Bar> ... </Bar>
  <ds:Signature>...</ds:Signature>
</Foo>

and then you encrypt Bar:

<Foo>
  <xenc:EncryptedData>....</xenc:EncryptedData>
  <ds:Signature>...</ds:Signature>
</Foo>


I don't see how it could be getting the two KeyInfo's confused. All the
Apache EncryptedData.getKeyInfo() does is return a data member, it
doesn't do any searching or resolution.

> My XML Signature
> element that is enveloped by the EncryptedData element contains it's own
> KeyInfo element that is being found by the call to
> "encryptedData.getKeyInfo();" (see
> http://www.koders.com/java/fidFC09B90248DEB9C9318CA3CAE0C9809BEEC94EEC.aspx#L1422)
>   

> So, to work around this issue, immediately after validating the enveloped
> signature, I am deleting the signature from the XML Document object
> "encryptedDataElement.removeChild(signatureNode);" so that it does not
> interfere with the decryption of the EncryptedData.
>   


Sound like you're trying to sign the EncryptedData itself?

<xenc:EncryptedData>
 <ds:Signature>...</ds:Signature>
</xenc:EncryptedData>


Note that you can't do that.   It's not schema valid, xenc:EncryptedData
(actually xenc:EncryptedType) doesn't have an open content model, you
can't just add arbitrary child elements.

And even if it did, I'm still not seeing how the Apache EncryptedData
would be pulling the KeyInfo out of the ds:Signature rather than it's
own immediate child.

--Brent

Re: Still stuck with problem. Re: Problem decrypting elements

Posted by rlyders <Ri...@Lyders.Net>.
Brent,
Your comments lead me to debug the XMCipher java class to find out that my
enveloped Signature is confusing the default decryption. My XML Signature
element that is enveloped by the EncryptedData element contains it's own
KeyInfo element that is being found by the call to
"encryptedData.getKeyInfo();" (see
http://www.koders.com/java/fidFC09B90248DEB9C9318CA3CAE0C9809BEEC94EEC.aspx#L1422)

So, to work around this issue, immediately after validating the enveloped
signature, I am deleting the signature from the XML Document object
"encryptedDataElement.removeChild(signatureNode);" so that it does not
interfere with the decryption of the EncryptedData.

This seems to be a fine workaround, but is there some way to tell the
EncryptedData element to skip over the XML signature element when looking
for the KeyInfo element? Or is there a better way to work with enveloped
signatures?


Brent Putman wrote:
> 
> So assuming you want to use an EncryptedKey handled (resolved) by the
> library at decryption time, you need to:
> 
> 1) When encrypting...add the
> EncryptedKey to *each* EncryptedData's KeyInfo.  (There are other ways
> to place and resolve EncryptedKey's but that's the default resolution
> mechanism supported by the library).
> 
> 

-- 
View this message in context: http://www.nabble.com/Problem-decrypting-elements-tf4699611.html#a14086621
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.

Re: Still stuck with problem. Re: Problem decrypting elements

Posted by rlyders <Ri...@Lyders.Net>.
Brent,
Your comments lead me to debug the XMCipher java class to find out that my
enveloped Signature is confusing the default decryption. My XML Signature
element that is enveloped by the EncryptedData element contains it's own
KeyInfo element that is being found by the call to
"encryptedData.getKeyInfo();" (see
http://www.koders.com/java/fidFC09B90248DEB9C9318CA3CAE0C9809BEEC94EEC.aspx#L1422)

So, to work around this issue, immediately after validating the enveloped
signature, I am deleting the signature from the XML Document object
"encryptedDataElement.removeChild(signatureNode);" so that it does not
interfere with the decryption of the EncryptedData.

This seems to be a fine workaround, but is there some way to tell the
EncryptedData element to skip over the XML signature element when looking
for the KeyInfo element? Or is there a better way to work with enveloped
signatures?


Brent Putman wrote:
> 
> So assuming you want to use an EncryptedKey handled (resolved) by the
> library at decryption time, you need to:
> 
> 1) When encrypting...add the
> EncryptedKey to *each* EncryptedData's KeyInfo.  (There are other ways
> to place and resolve EncryptedKey's but that's the default resolution
> mechanism supported by the library).
> 
> 

-- 
View this message in context: http://www.nabble.com/Problem-decrypting-elements-tf4699611.html#a14086621
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.

Re: Still stuck with problem. Re: Problem decrypting elements

Posted by Wolfgang Freis <wo...@yale.edu>.
Thanks, Brent.

Your response identified the problem.  I hadn't added the KeyInfo to each
EncryptedData, but only once.  So, as long as there was only one encrypted
element, it worked but threw an exception if there were more.  Works like a
charm now.

Thanks again,
Wolfgang


>
> 1) When encrypting, make sure to generate an EncryptedKey from the
> symmetric data encryption key using the desired key encryption key.  If
> you're using asymmetric keys for key transport, then the latter would be
> the public key half of the recipient's key pair.  Then add the
> EncryptedKey to *each* EncryptedData's KeyInfo.  (There are other ways
> to place and resolve EncryptedKey's but that's the default resolution
> mechanism supported by the library).
>

Re: Still stuck with problem. Re: Problem decrypting elements

Posted by Brent Putman <pu...@georgetown.edu>.
Hi Wolfgang,
Since it's falling into the code branch in decryptToByteArray(Element)
that indicates that you did not specify a data decryption key to
XMLCipher#init(int, Key), then I assume you want to use an EncryptedKey
to carry the carry the data decryption key and that you want the library
to handle resolving that.  (If that's not the case then you should
specify a non-null Key to XMLCipher#init(int, Key) when decrypting,
possibly by resolving it yourself "manually" from the EncryptedData's
KeyInfo or from other contextual information, e.g. knowledge of the sender.)

The exception and log error you're seeing on decryption would be caused
by either the EncryptedData's KeyInfo being null, or the failure of the
registered KeyResolvers to resolve the data decryption key from the
EncryptedData's KeyInfo.

So assuming you want to use an EncryptedKey handled (resolved) by the
library at decryption time, you need to:

1) When encrypting, make sure to generate an EncryptedKey from the
symmetric data encryption key using the desired key encryption key.  If
you're using asymmetric keys for key transport, then the latter would be
the public key half of the recipient's key pair.  Then add the
EncryptedKey to *each* EncryptedData's KeyInfo.  (There are other ways
to place and resolve EncryptedKey's but that's the default resolution
mechanism supported by the library).

2) When decrypting, make sure to set the key encryption key (really
better called the key decryption key) using XMLCipher#setKEK(Key).  If
you're using asymmetric keys for key transport, then that is going to be
the private key half of the recipient's key pair.

If that still doesn't clear it up, you might want to post some of the
code that shows how you are both encrypting and decrypting in the
failure case.  It does sound like you are just omitting something
simple.  Not sure why it's working for you in some cases and not others.

Are you doing things differently with key handling/resolution in the
single element vs. multiple element cases?  Also, since it sounded like
you were just running in an IDE: are you by chance reusing the same
XMLCipher instance across both the encryption and decryption
operations?  It's possible that there might be a bug that might cause
unpredictable results due to internal state that's being tracked or
something.  Just speculating.

--Brent



Wolfgang Freis wrote:
> Hello all,
>
> I'm still stuck with this problem and don't know what else to try.  I'm
> basically using the sample code with some modifications to handle multiple
> elements.
>
> Encryption works fine with child elements or contents both when a single or
> multiple elements with the same name are encrypted.  However, decryption
> only works when there is a single element or content node to decrypt.  If
> there are more, an exception is thrown:
>
>   Oct 30, 2007 2:28:18 PM org.apache.xml.security.encryption.XMLCipher
>     decryptToByteArray
>   SEVERE: XMLCipher::decryptElement called without a key and unable to
>     resolve Exception in thread "main"
>     org.apache.xml.security.encryption.XMLEncryptionException: No Key
>     Encryption Key loaded and cannot determine using key resolvers
>
> Stepping through the code in my IDE, I can see that the key gets loaded. 
> What is missing, though, is the KeyInfo.  Method
> decryptToByteArray(Element element) in class XMLCipher tries to get the
> KeyInfo:
>
>    KeyInfo ki = encryptedData.getKeyInfo();
>
> When ki gets evaluated it is null, however, and the exception is thrown. 
> Again, it only happens when multiple elements are encrypted.
>
> Obviously, I'm missing something here but neither the documentation nor the
> email archive have given me any clues on how to solve this.  Any help will
> be greatly appreciated.
>
> Thanks,
> Wolfgang
>
>
>
>
>
>
>
> Quoting Wolfgang Freis <wo...@yale.edu>:
>
>   
>> Hello,
>>
>> I'm new to working with XML encryption and have come across a problem.
>> Neither the email archive nor have brought me closer to a solution.
>>
>> Google Starting with the sample files Encrypter.java and Decrypter.java
>> I
>> have managed to en- and decrypt elements by name.  However, decryption
>> works only when there is only one element encrypted.  If I encrypt
>> multiple elements by name--as siblings or as children of different
>> parents--the decryptor throws an exception:
>>
>> Oct 26, 2007 3:56:45 PM org.apache.xml.security.encryption.XMLCipher
>>     decryptToByteArray
>> SEVERE: XMLCipher::decryptElement called without a key and unable to
>>     resolve
>> Exception in thread "main"
>> org.apache.xml.security.encryption.XMLEncryptionException:
>>     No Key Encryption Key loaded and cannot determine using key
>> resolvers
>>
>> My IDE tells that the key file was loaded and, as I said, the decryption
>> works fine when only one element of the XML is encrypted.  I tried
>> decrypting by iterating over the NodeList returned from
>> getElementsByTagNameNS(namespaceURI, localName) and trying to just grab
>> the first item.  Both generate the same exception.  From what I can see
>> in
>> the IDE is that the method decryptToByteArray(Element element) called
>> from
>> doFinal( document, encryptedDataElement) throws the exception, but it
>> escapes me why that happens.
>>
>> Any help will be greatly appreciated.
>>
>> Thanks,
>> Wolfgang
>>
>>
>>
>>     
>
>