You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Christopher Schultz <ch...@christopherschultz.net> on 2023/01/31 18:51:49 UTC
Issues playing around with digest.sh
All,
I was trying to run digest.sh today with some potentially unusual
arguments and it was behaving in ways I didn't expect.
First, I wanted to get it to generate PBKDF2 hashes, and so I tried the
most obvious thing I could think of:
$ digest.sh -a 'PBKDF2' 'secret'
I got this error output:
Jan 31, 2023 11:11:59 AM org.apache.tomcat.util.IntrospectionUtils
setProperty
WARNING: IntrospectionUtils: InvocationTargetException for class
org.apache.catalina.realm.MessageDigestCredentialHandler algorithm=PBKDF2)
java.lang.reflect.InvocationTargetException
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at
org.apache.tomcat.util.IntrospectionUtils.setProperty(IntrospectionUtils.java:70)
at
org.apache.tomcat.util.IntrospectionUtils.setProperty(IntrospectionUtils.java:46)
at org.apache.catalina.realm.RealmBase.main(RealmBase.java:1492)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at org.apache.catalina.startup.Tool.main(Tool.java:230)
Caused by: java.security.NoSuchAlgorithmException: PBKDF2 MessageDigest
not available
at java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
at java.base/java.security.Security.getImpl(Security.java:700)
at
java.base/java.security.MessageDigest.getInstance(MessageDigest.java:183)
at
org.apache.tomcat.util.security.ConcurrentMessageDigest.init(ConcurrentMessageDigest.java:118)
at
org.apache.catalina.realm.MessageDigestCredentialHandler.setAlgorithm(MessageDigestCredentialHandler.java:90)
... 12 more
Jan 31, 2023 11:11:59 AM org.apache.tomcat.util.IntrospectionUtils
setProperty
WARNING: IntrospectionUtils: InvocationTargetException for class
org.apache.catalina.realm.SecretKeyCredentialHandler algorithm=PBKDF2)
java.lang.reflect.InvocationTargetException
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at
org.apache.tomcat.util.IntrospectionUtils.setProperty(IntrospectionUtils.java:70)
at
org.apache.tomcat.util.IntrospectionUtils.setProperty(IntrospectionUtils.java:46)
at org.apache.catalina.realm.RealmBase.main(RealmBase.java:1492)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at org.apache.catalina.startup.Tool.main(Tool.java:230)
Caused by: java.security.NoSuchAlgorithmException: PBKDF2
SecretKeyFactory not available
at
java.base/javax.crypto.SecretKeyFactory.<init>(SecretKeyFactory.java:118)
at
java.base/javax.crypto.SecretKeyFactory.getInstance(SecretKeyFactory.java:164)
at
org.apache.catalina.realm.SecretKeyCredentialHandler.setAlgorithm(SecretKeyCredentialHandler.java:56)
... 12 more
secret:9e667cf452bd6eddc71d9613605a2dd75527d8fb9aa2b97c918d7aa3ae9ef995$20000$69049fa7dec3dc9761092ff7b4d39ac5f7b798ec
Woah. Two failures and a success? Interesting.
The two failures tell me that both the MessageDigestCredentialHandler
and the SecretKeyCredentialHandler both failed to set that algorithm,
because it doesn't exist (in either one). Super ugly and scary, but
technically correct.
What about that "success" I got instead? Well... it includes 2 $ symbols
with 20000 in between them, indicating that 20000 rounds of *something*
was done. The default number of rounds for SecretKeyCredentialHandler is
20000 so it's likely that I got the default algorithm for that class,
which is PBKDF2WithHmacSHA1.
Honestly, I should probably just have gotten an error saying "couldn't
decide what to do" instead of getting a weird default, especially when
the default algorithm for RealmBase (which really does this work) is
single-iteration, non-salted SHA-215.
These switching-defaults happen because of the way the handler is
chosen. The code is here in RealmBase:
CredentialHandler handler = null;
if (handlerClassName == null) {
for (Class<? extends DigestCredentialHandlerBase> clazz :
credentialHandlerClasses) {
try {
handler = clazz.getConstructor().newInstance();
if (IntrospectionUtils.setProperty(handler,
"algorithm", algorithm)) {
break;
}
} catch (ReflectiveOperationException e) {
// This isn't good.
throw new RuntimeException(e);
}
}
} else {
try {
Class<?> clazz = Class.forName(handlerClassName);
handler = (DigestCredentialHandlerBase)
clazz.getConstructor().newInstance();
IntrospectionUtils.setProperty(handler, "algorithm",
algorithm);
} catch (ReflectiveOperationException e) {
throw new RuntimeException(e);
}
}
if (handler == null) {
throw new RuntimeException(new
NoSuchAlgorithmException(algorithm));
}
If we get (swallowed) errors while trying to find out which
CredentialHandler should performing the password mutation, then we just
always use the last one we tried. That effectively changes the default
CredentialHandler to the last one in the search list. I think we should
set handler=null in the case where the algorithm failed to be set.
Okay, what about specifying the correct algorithm name?
$ digest.sh -a 'PBKDF2WithHmacSHA1' 'secret'
Jan 31, 2023 1:31:28 PM org.apache.tomcat.util.IntrospectionUtils
setProperty
WARNING: IntrospectionUtils: InvocationTargetException for class
org.apache.catalina.realm.MessageDigestCredentialHandler
algorithm=PBKDF2WithHmacSHA1)
java.lang.reflect.InvocationTargetException
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at
org.apache.tomcat.util.IntrospectionUtils.setProperty(IntrospectionUtils.java:70)
at
org.apache.tomcat.util.IntrospectionUtils.setProperty(IntrospectionUtils.java:46)
at org.apache.catalina.realm.RealmBase.main(RealmBase.java:1492)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at org.apache.catalina.startup.Tool.main(Tool.java:230)
Caused by: java.security.NoSuchAlgorithmException: PBKDF2WithHmacSHA1
MessageDigest not available
at java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
at java.base/java.security.Security.getImpl(Security.java:700)
at
java.base/java.security.MessageDigest.getInstance(MessageDigest.java:183)
at
org.apache.tomcat.util.security.ConcurrentMessageDigest.init(ConcurrentMessageDigest.java:118)
at
org.apache.catalina.realm.MessageDigestCredentialHandler.setAlgorithm(MessageDigestCredentialHandler.java:90)
... 12 more
secret:548a028e43ca1cd1690a445e0c15fc255ad4d9626d7ca9724c4ff20a9354e7b6$20000$8d79de026f8ee9262a6f85d295338c41d1657181
One failure and one success. This time, we actually got exactly what we
wanted, but we still got a really scary error message. Okay, it' a
warning, but it still looks scary. Would you use that output for
authentication if you got a stack trace like that?
I'm not sure how best to change that. I can think of two obvious
potential solutions:
1. Modify the logger at runtime to suppress warnings
2. Modify IntrospectionUtils.setProperty to allow errors to be
suppressed instead of logged as error
I'm not sure if #1 is easily doable given lots of ways to configure
logging and I'm not sure there is any appetite to do #2. We may be
unable to suppress the warnings, but maybe we can add a message later
that says "you can ignore any warnings you see above about missing
algorithms".
The final problem I see is, unfortunately, a nasty quoting problem. I
discovered that PBKDF2 has a collision problem with passwords longer
than the core hashing algorithm's block size. In those cases, the
original password is hashed one time all by itself, and then the result
is fed into the iterated salted algorithm. That means that any password
that is longer than e.g. 20 bytes for SHA1 will have the exact same
PBKDF2 output as the SHA1 hash of that password does.
I wanted to test this using the sample inputs provided on the Wikipedia
page for PBKDF2[1], which is "eBkXQTfuBqp'cTcar&g*" (without the
double-quotes). Note that it contains a single quote.
I could not get digest.sh to accept that password on the command-line
regardless of the way I quoted it.
digset.sh calls tool-wrapper.sh using "@*". This should be correct.
tool-wrapper.sh calls eval exec ... "@*" which is where I think the
problem is. I think eval "unrwaps" the "@*" and the resulting
command-line has a bare value at the end. In this case, it's a weird
thing that includes a trifecta of problematic characters on the
command-line: a single quote, an ampersand, and an asterisk.
I'm not sure how to get around that other than replacing "eval exec"
with "exec" or maybe removing both of them. I don't know the history of
decisions that led to the use of "eval exec" but I'm sure there are Good
Reasons for those things to be in there. If they have to stay, I think
we have a definite limitation of the digest.sh tool in that some
characters simply cannot be used in command-line parameters, which is a
real shame.
-chris
[1] https://en.wikipedia.org/wiki/PBKDF2
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Issues playing around with digest.sh
Posted by Mark Thomas <ma...@apache.org>.
On 31/01/2023 18:51, Christopher Schultz wrote:
<snip/>
> $ digest.sh -a 'PBKDF2' 'secret'
<snip/>
> If we get (swallowed) errors while trying to find out which
> CredentialHandler should performing the password mutation, then we just
> always use the last one we tried. That effectively changes the default
> CredentialHandler to the last one in the search list. I think we should
> set handler=null in the case where the algorithm failed to be set.
+1
> Okay, what about specifying the correct algorithm name?
>
> $ digest.sh -a 'PBKDF2WithHmacSHA1' 'secret'
<snip/>
> One failure and one success. This time, we actually got exactly what we
> wanted, but we still got a really scary error message. Okay, it' a
> warning, but it still looks scary. Would you use that output for
> authentication if you got a stack trace like that?
>
> I'm not sure how best to change that. I can think of two obvious
> potential solutions:
>
> 1. Modify the logger at runtime to suppress warnings
> 2. Modify IntrospectionUtils.setProperty to allow errors to be
> suppressed instead of logged as error
>
> I'm not sure if #1 is easily doable given lots of ways to configure
> logging and I'm not sure there is any appetite to do #2. We may be
> unable to suppress the warnings, but maybe we can add a message later
> that says "you can ignore any warnings you see above about missing
> algorithms".
I think #1 is possible. The IntrospectionUtils logger is always going to
be obtained via LogFactory so this should work:
// Load the class to create the logger
IntrospectionUtils.escape(null);
// Disable warning messages
LogManager.getLogManager().getLogger(IntrospectionUtils.class.getName()).setLevel(Level.SEVERE);
> The final problem I see is, unfortunately, a nasty quoting problem.
<snip/>
> I wanted to test this using the sample inputs provided on the Wikipedia
> page for PBKDF2[1], which is "eBkXQTfuBqp'cTcar&g*" (without the
> double-quotes). Note that it contains a single quote.
>
> I could not get digest.sh to accept that password on the command-line
> regardless of the way I quoted it.
>
> digset.sh calls tool-wrapper.sh using "@*". This should be correct.
>
> tool-wrapper.sh calls eval exec ... "@*" which is where I think the
> problem is. I think eval "unrwaps" the "@*" and the resulting
> command-line has a bare value at the end. In this case, it's a weird
> thing that includes a trifecta of problematic characters on the
> command-line: a single quote, an ampersand, and an asterisk.
>
> I'm not sure how to get around that other than replacing "eval exec"
> with "exec" or maybe removing both of them. I don't know the history of
> decisions that led to the use of "eval exec" but I'm sure there are Good
> Reasons for those things to be in there. If they have to stay, I think
> we have a definite limitation of the digest.sh tool in that some
> characters simply cannot be used in command-line parameters, which is a
> real shame.
`eval exec` is the result of trying to handle various oddities in
JAVA_OPTS and CATALINA_OPTS. I suspect we'll need to change something in
tool-wrapper to fix this but I don't immediately see a solution.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org