You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@santuario.apache.org by Martin Pirker <Ma...@iaik.tugraz.at> on 2006/01/25 10:29:42 UTC
base64 elements linebreak
Hello...
While doing interoperability experiments with XML documents
containing digital signatures created with the Apache xmlsec
project libraries (=the libraries contained in the Java WSDP
package), I noticed a difference to other providers
output.
Base64 coded components, e.g. the Modulus of a RSA key or
the SignatureValue itself, get wrapped at column 76.
Why is that?
Digging in the source, e.g. XMLSignature.java, there is:
|private void setSignatureValueElement(byte[] bytes)
|if (base64codedValue.length() > 76) {
| base64codedValue = "\n"+base64codedValue+"\n";
|}
..of course it wraps.
But why does the output of other libs not linebreak?
Is this decision backed by some standard?
RFC3548, Section 2.1, Line feeds in encoded data,
explains that the 76 come from MIME, not base64 itself.
Whereas
http://www.w3.org/TR/xmlschema-2/#base64Binary
notes:
"For compatibility with older mail gateways, [RFC
2045] suggests that base64 data should have lines
limited to at most 76 characters in length. This
line-length limitation is not mandated in the lexical
forms of base64Binary data and must not be enforced
by XML Schema processors."
While wrapping at 76 appears to do neither good nor
harm in the XML world, I still "feel" this is "ugly".
Large autogenerated XML docs usually contain no
linebreaks and appear on one line.
Except when you sign them with Apache xmlsec, now
they get a few extra linebreaks...
I personally would prefer no linebreaks, but maybe
there was a good reason it was coded that way?
Thanks,
Martin
Re: base64 elements linebreak
Posted by Berin Lautenbach <be...@wingsofhermes.org>.
Tech Rams wrote:
> Yes, that is true. This flag does not work if there are NL chars in the
> message (anywhere). Thus, you will have to scan the message to find out
> if there are NL chars - but the input has to follow some spec - either
> they do not have any new lines at all, per
> _http://www.w3.org/TR/xmlschema-2/#base64Binary_ - or they have new
> lines within 76 chars, per RFC2045.
Unfortunately bitter experience says that you have to support people who
do wierd things outside the 76 chars. I also don't want to pre-scan
base64 data, which could be megabytes. So I try to avoid using OpenSSL
anywhere that it might be an issue now.
Cheers,
Berin
RE: base64 elements linebreak
Posted by Scott Cantor <ca...@osu.edu>.
> So, for example, if the original base64 encoded data
> comes through SMTP, we will have to decode and encode
> it again if we have to put the same value in an XML?
No, precisely the opposite. Anything you get that's legal base64, whatever
the line length, is "legal" XML, and moreover is valid with respect to the
rules for base64Binary in XSD. So saying that it should look a certain way
is only advisory.
In other words, xmlsec producing the line feeds is legal. That's all I was
trying to say. Requiring them would not be, obviously.
I must have said something that confused you or that I didn't say clearly. I
wasn't trying to argue anything else.
> What I am driving at is that having new lines in
> base64 encoded data is not restricted by any spec - in
> fact it is deemed necessary by at least one spec.
> And also that you can make your decoder flexible by
> checking and setting the flag as I mentioned earlier.
Sure. All I was saying in the thread was that one reason why the linefeeds
were useful was because that flag was either not known, not implemented,
etc, so without them, using OpenSSL's decoder was a problem. And the
workaround was to not use OpenSSL's decoder, which is really the better
solution...yeah, the flag's there, but why bother when you can just do it in
one pass without examining the data first?
> But normalization is a real problem for signature
> validation, as you mentioned earlier - looks like
> whitespace facet for signed content must always be
> preserve - particularly if you have schema validators
> before signature verification.
Correct, and of course this isn't allowed by XSD, which requires collapse
for base64Binary. So you have to turn the normalization off to get it to
work.
In practice, what makes things tend to work is that schema validation and
signature-oriented applications aren't all that overlapping in practice,
something I had to learn the hard way.
-- Scott
RE: base64 elements linebreak
Posted by Tech Rams <te...@yahoo.com>.
--- Scott Cantor <ca...@osu.edu> wrote:
>
> There is no spec in XML-land that says base64
> content uses any particular
> line length.
>
So, for example, if the original base64 encoded data
comes through SMTP, we will have to decode and encode
it again if we have to put the same value in an XML?
>
> That's advisory only, and XML schema mostly deals
> with what normalization
> results in, not with the raw XML is.
>
I guess I am failing to understand how 'advisory' can
be construed as 'do not do it'. XML-schema spec says
'This line-length limitation is not mandated in the
lexical forms of base64Binary data and must not be
enforced by XML Schema processors.' - which I
interpret as 'RFC2045 is NOT a must - so do not
enforce it' - rather than as 'RFC2045 is wrong - make
sure there are no new lines'.
The same spec (XML-Schema) also explains that the
whitespace facet of base64Binary is fixed to
'collapse' - so a schema validator will indeed convert
new lines to spaces. But this is still OK - as all the
new lines will be converted to spaces - the logic for
using OpenSSL's flag I gave earlier will still work
(spaces are neglected by base64 decoders).
What I am driving at is that having new lines in
base64 encoded data is not restricted by any spec - in
fact it is deemed necessary by at least one spec.
And also that you can make your decoder flexible by
checking and setting the flag as I mentioned earlier.
But normalization is a real problem for signature
validation, as you mentioned earlier - looks like
whitespace facet for signed content must always be
preserve - particularly if you have schema validators
before signature verification.
-rams
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
RE: base64 elements linebreak
Posted by Scott Cantor <ca...@osu.edu>.
> Yes, that is true. This flag does not work if there are NL
> chars in the message (anywhere). Thus, you will have to scan
> the message to find out if there are NL chars - but the input
> has to follow some spec -
There is no spec in XML-land that says base64 content uses any particular
line length.
> either they do not have any new lines at all, per
> http://www.w3.org/TR/xmlschema-2/#base64Binary
That's advisory only, and XML schema mostly deals with what normalization
results in, not with the raw XML is.
-- Scott
Re: base64 elements linebreak
Posted by Tech Rams <te...@yahoo.com>.
Yes, that is true. This flag does not work if there are NL chars in the message (anywhere). Thus, you will have to scan the message to find out if there are NL chars - but the input has to follow some spec - either they do not have any new lines at all, per http://www.w3.org/TR/xmlschema-2/#base64Binary - or they have new lines within 76 chars, per RFC2045.
-rams
Berin Lautenbach <be...@wingsofhermes.org> wrote:
I tried using this at one point. It caused yet more problems in that
something *with* a line break at the correct point broke. So if you
have something that is non-spec with where it puts the line break, you
are out of luck.
It also means you have to pre-scan the base64 text to work out what you
are going to do - which is just plain ugly.
Cheers,
Berin
---------------------------------
Do you Yahoo!?
With a free 1 GB, there's more in store with Yahoo! Mail.
problem with Algorithm ecdsa-sha1
Posted by Markus Lindner <ma...@catbull.com>.
Hi there,
I use xmlsecurity to verify Signatures (dsig) in xml-files.
I got some _new_ files, which via xmlsecurity do not work the way I expected:
org.apache.xml.security.signature.XMLSignatureException: The requested
algorithm http://www.w3.org/2001/04/xmldsig-more#ecdsa-sha1 does not
exist. Original Message was: null
Original Exception was java.lang.NullPointerException
at
org.apache.xml.security.algorithms.SignatureAlgorithm.<init>(Unknown
Source)
at org.apache.xml.security.signature.SignedInfo.<init>(Unknown
Source)
at org.apache.xml.security.signature.XMLSignature.<init>(Unknown
Source)
I use bouncycastle as my provider. I add the provider dynamically before
verifying.
When I list all the algorithms, I got:
[long list left out]
Signatures:
[long list left out]
SHA1WITHECNR
OID.1.2.840.113549.1.1.5
SHA512withRSA
ECDSA
SHA512WITHRSAENCRYPTION
[long list left out]
I think that "http://www.w3.org/2001/04/xmldsig-more#ecdsa-sha1" should
point to ECDSA.
I use xmlsecurity-1.3 (just the jar from the web
http://xml.apache.org/security/dist/java-library/ )
I am aware of the xalan/endorsed problem. I tested on linux java 1.4/1.5
and winxp 1.4/1.5.
I am not sure if xmlsecurity can handle this anyway...just wanted to ask
what is the problem here...
greetings
m.
--
"0x07: Signature not present. Press any key."
RE: base64 elements linebreak
Posted by Scott Cantor <ca...@osu.edu>.
> I am not sure if you are implying that normalization removes
> line breaks.
Yep. Schema normalization does exactly that.
> AFAIK, normalization just 'joins' adjacent text
> nodes - not dealing with new lines in any way. (I am
> referring to the doc.normalize() call. Please let me know if
> you are talking about something else.)
Yep, I am.
-- Scott
RE: base64 elements linebreak
Posted by Tech Rams <te...@yahoo.com>.
I am not sure if you are implying that normalization removes line breaks. AFAIK, normalization just 'joins' adjacent text nodes - not dealing with new lines in any way. (I am referring to the doc.normalize() call. Please let me know if you are talking about something else.)
-rams
Scott Cantor <ca...@osu.edu> wrote:
> It also means you have to pre-scan the base64 text to work out what you
> are going to do - which is just plain ugly.
Yeah. The bottom line is that anything that can't consume something with or
without breaks is non-compliant, so clearly OpenSSL is borderline because
the flag is not attractive to use, and anything that breaks because Apache
code *is* producing them is also broken.
This all gets more complex if you're including schema validation, because of
data normalization, but Xerces-C 2.7 has finally dealt with that I believe.
In 2.6, they created a double-whammy by breaking the signature if you
normalized, but breaking the base64 decoding if you didn't. That was fun.
-- Scott
---------------------------------
Do you Yahoo!?
With a free 1 GB, there's more in store with Yahoo! Mail.
RE: base64 elements linebreak
Posted by Scott Cantor <ca...@osu.edu>.
> It also means you have to pre-scan the base64 text to work out what you
> are going to do - which is just plain ugly.
Yeah. The bottom line is that anything that can't consume something with or
without breaks is non-compliant, so clearly OpenSSL is borderline because
the flag is not attractive to use, and anything that breaks because Apache
code *is* producing them is also broken.
This all gets more complex if you're including schema validation, because of
data normalization, but Xerces-C 2.7 has finally dealt with that I believe.
In 2.6, they created a double-whammy by breaking the signature if you
normalized, but breaking the base64 decoding if you didn't. That was fun.
-- Scott
Re: base64 elements linebreak
Posted by Berin Lautenbach <be...@wingsofhermes.org>.
Tech Rams wrote:
> BTW, OpenSSL has a flag for such cases - BIO_FLAGS_BASE64_NO_NL. Setting
> this flag would allow data with no line breaks to be decoded.
I tried using this at one point. It caused yet more problems in that
something *with* a line break at the correct point broke. So if you
have something that is non-spec with where it puts the line break, you
are out of luck.
It also means you have to pre-scan the base64 text to work out what you
are going to do - which is just plain ugly.
Cheers,
Berin
RE: base64 elements linebreak
Posted by Scott Cantor <ca...@osu.edu>.
> BTW, OpenSSL has a flag for such cases -
> BIO_FLAGS_BASE64_NO_NL. Setting this flag would allow data
> with no line breaks to be decoded.
How new is the flag? I ask because it seemed to be a problem in so many
places for quite a while.
-- Scott
RE: base64 elements linebreak
Posted by Tech Rams <te...@yahoo.com>.
BTW, OpenSSL has a flag for such cases - BIO_FLAGS_BASE64_NO_NL. Setting this flag would allow data with no line breaks to be decoded.
-rams
Scott Cantor <ca...@osu.edu> wrote:
> While wrapping at 76 appears to do neither good nor
> harm in the XML world, I still "feel" this is "ugly".
>
> Large autogenerated XML docs usually contain no
> linebreaks and appear on one line.
> Except when you sign them with Apache xmlsec, now
> they get a few extra linebreaks...
>
> I personally would prefer no linebreaks, but maybe
> there was a good reason it was coded that way?
Try decoding those single line blobs with OpenSSL. Boom. For a lot of apps,
that's a serious problem.
C++ xmlsec had to recode around this and stop using OpenSSL calls to load
certificates and such, but the issue still crops up.
-- Scott
---------------------------------
Do you Yahoo!?
With a free 1 GB, there's more in store with Yahoo! Mail.
RE: base64 elements linebreak
Posted by Scott Cantor <ca...@osu.edu>.
> While wrapping at 76 appears to do neither good nor
> harm in the XML world, I still "feel" this is "ugly".
>
> Large autogenerated XML docs usually contain no
> linebreaks and appear on one line.
> Except when you sign them with Apache xmlsec, now
> they get a few extra linebreaks...
>
> I personally would prefer no linebreaks, but maybe
> there was a good reason it was coded that way?
Try decoding those single line blobs with OpenSSL. Boom. For a lot of apps,
that's a serious problem.
C++ xmlsec had to recode around this and stop using OpenSSL calls to load
certificates and such, but the issue still crops up.
-- Scott