You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Alex Scheel <as...@redhat.com> on 2019/12/05 20:18:57 UTC
Tomcat - No Fork for debugging?
Hi all,
I've tried searching to no avail.
I'm working on a(nother) SSL adapter. However, I've had some issues with
it. There's a native component and I'm trying to tease apart its relationship
with why the client won't handshake. The stack traces aren't overly helpful
and I'd love to attach gdb to this and set a few specific environment
variables.
Query:
Does Tomcat have a mode where it won't fork to a different user and will run
with a limited number of threads? That'd greatly improve my ability to debug.
Something similar to `radiusd -X` or `sshd -d`?
Rationale:
NSS has support for logging calls to its PKCS#11 interface to a file, based
on the presence of environment variables. When I set these environment variables
and directly call the JVM to start Tomcat:
# java -classpath $CLASSPATH $FLAGS org.apache.catalina.startup.Bootstrap start
I see it logging calls when the JDK starts up, but when I hit it with wget on
the TLS port, the resulting PKCS#11 calls aren't logged. When launching in gdb,
I get an error about /sbin/nologin doesn't understand the -c option, which to
me says that Bootstrap is trying to fork and create a new shell (I'm running as
root in a VM and it wants to launch as the tomcat user), dropping my environmental
variables I want.
Ideally (for debugging) I'd like to simplify this. Is there a more direct
entry route I can use perhaps?
- Alex
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat - No Fork for debugging?
Posted by Alex Scheel <as...@redhat.com>.
----- Original Message -----
> From: "Christopher Schultz" <ch...@christopherschultz.net>
> To: users@tomcat.apache.org
> Sent: Friday, December 6, 2019 12:41:23 PM
> Subject: Re: Tomcat - No Fork for debugging?
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Alex,
>
> On 12/6/19 10:57, Alex Scheel wrote:
> > Apologies, this reply got away from me... :-)
>
> Not at all. This is very helpful.
>
> > Perhaps grab a fresh $BEVERAGE before reading.
>
> It's before noon, so I grabbed a fresh espresso.
>
> :)
>
~snip older context~
> >> RHEL ships OpenJDK and upstream itself doesn't do the FIPS
> >> certification; it is Red Hat that does the certification of NSS
> >> on RHEL. JDK itself, under the FIPS configuration above, now does
> >> no crypto. All the crypto is done via PKCS#11 to NSS. So there's
> >> nothing (in OpenJDK) to certify any more. Unlike in the non-FIPS
> >> mode above, there's lots of crypto that would need to be
> >> certified.
>
> Aah, okay. This makes more sense to me, now. At this point, the JSSE
> provider is just a pass-through to NSS, just like the JSSE wrapper for
> OpenSSL.
Yep! And that's essentially how JSS behaves as well. Different wrappers
with different goals by different teams. Same underlying crypto.
>
> >> Oracle, in their JDK, might ship their own PKCS#11 provider
> >> and/or certify NSS, I don't know. I know IBM ships their own FIPS
> >> crypto provider, and probably the next most popular is
> >> BouncyCastle, but I thought that might be losing popularity
> >> because they might've stopped FIPS certifying it, who knows.
>
> IMHO FIPS is useless at this point unless you are required to use a
> FIPS-certified module, which many people are unfortunately required to
> use. The reason I think it's useless is because FIPS mandates the use
> of certain algorithms and some of them shouldn't be used.
While I personally agree, there's still a number of people who need it
or claim they want it. But I will say it is getting better, and oh, idk, by
2025 or so we should have FIPS 140-5 (with X25519 allowed!) and a sane set
of cipher suites and allowed primitives (between SP800-56B rev 2 and a few
other publications).
By then... who knows what we'll have outside of FIPS. :-)
> >> But my key point is there's no "OpenJDK FIPS Certificate" on the
> >> FIPS page:
> >
> >> https://csrc.nist.gov/projects/cryptographic-module-validation-progra
> m/validated-modules/search
> >
> >> (Search Java)
> >
> >> But there is one for NSS ("NSS" - and then its "NSS Cryptographic
> >> Module" on the results page).
>
> Yep. I saw that Oracle has FIPS-certified modules, but they aren't a
> part of the standard Java distribution (and certainly not part of
> OpenJDK). I guess you get what you pay for.
>
> >> So yes, the SunJSSE provider is still used but the key
> >> differences are:
> >
> >> 1) The crypto is done by SunPKCS11-NSS-FIPS provider, 2) The NSS
> >> that is used is FIPS certified, 3) It places additional
> >> restrictions (more about that below).
> >
> >> If you don't really care 2 (the token "Certificate") and just
> >> want to test stuff, 1 and 3 work on any distro (e.g., Ubuntu and
> >> Fedora), and NSS everywhere understands FIPS mode.
> >
> >> Also the SunPKCS11 / SunPKCS11-NSS-FIPS providers are limited to
> >> Linux, iirc.
>
> Well, who would bother with NSS on Windows?
Doesn't Mozilla still ship Firefox with NSS as the provider on
Windows? Don't get me wrong, OpenSSL has definitely won the server
space especially on Linux, but NSS still has a non-zero presence of
client software just by virtue of Firefox and CentOS / RHEL 7 boxes
(things like curl use it there).
Also I know of some upstream consumers of JSS who run it on Mac
and Windows (and thus build NSS on both).
It can be done.
~snip long backtrace~
> Aha. "What's wrong with wrapping the KeyManager?"
>
> Answer: it doesn't work: KeyManagementException
>
> I guess they don't want any funny business.
>
> I'll have to look at how the KeyManager is built/wrapped. I didn't
> write that code so I'm not too familiar with it. I know we need to
> wave our hands around certain things due to the flexibility Tomcat
> allows for certificate-selection, etc.
>
> When you get that error, is the object of type
> org/apache/tomcat/util/net/jsse/JSSEKeyManager?
Yessir.
jdb: stop at JSSEKeyManager:<init> and a couple of step ups / steps
later and you can see it get passed to the init of SSLEngine that is
using the FIPS SunJSSE and the exception thrown.
(And you can validate the type of the passed KeyManager[]'s -- there's
only one element in the array).
> I wonder if we are jumping the gun and always using that KeyManager
> even when it's not strictly necessary.
Well. What your KeyManager does (and one of the things I've not liked
about the actual KeyManager interface because it doesn't do) is that it
gives control of the certificate to the user :-) The _user_ configures
a certificate they want to use and they know they're getting that
certificate. The default KeyManager interface (whether X509KeyManager
or X509ExtendedKeyManager) leaves selection of the certificate to the
SSLEngine / KeyManager itself, based on clues the SSLEngine gives it.
That's giving control to the devloper of some library somewhere in your
stack. (I understand why it exists, to facilitate SNI/eSNI, but...)
So in a messy NSS DB, potentially with lots of certificates with lots
of subject names, you're not exactly sure which certificate you'd get,
under the strict KeyManager interface. IMO, Tomcat's JSSE KeyManager
wrapper fixes that problem. And that's why wrapping KeyManagers is
necessary while not limiting SNI potential.
s/NSS DB/jks/ I guess in the above for a more... popular take. :-)
>
> >> My config roughly looks like:
> >
> >> <Connector name="Secure" port="8443" scheme="https" secure="true"
> >> SSLEnabled="true" keyAlias="localhost"
> >> keystorePass="keystorePass" keystoreType="PKCS11"
> >> keystoreProvider="SunPKCS11-NSS-FIPS"
> >> keyManagerAlgorithm="SunX509" truststoreAlgorithm="PKCS11"
> >> truststoreProvider="SunJSSE" truststoreType="SunX509"
> >> truststorePassword="truststorePassword" SSLProtocol="TLSv1.2"
> >> maxThreads="1" />
> >
> >> (Yeah, that's messy -- but it illustrates the point -- you can
> >> probably remove a number of those elements -- *including*
> >> keyAlias, and it'll still fail).
> >
> >> This is because, for all KeyManagers, Tomcat wraps it its own
> >> KeyManager:
> >
> >> https://github.com/apache/tomcat/blob/master/java/org/apache/tomcat/u
> til/net/SSLUtilBase.java#L378
> >
> >> Probably the minimum untested change that I'd have proposed
> >> would be to wrap it in an if block like you have ealier:
> >
> >> https://github.com/apache/tomcat/blob/master/java/org/apache/tomcat/u
> til/net/SSLUtilBase.java#L195
>
> Sounds like the same as my conclusion. I wouldn't want to make that
> change myself without understanding the other implications.
Right. But I also kinda want to leave it alone and use it to push back
and say hey, this is how its used in the wild.
>
> >> Now the core of the _why_ is because that's how NSS behaves. In
> >> FIPS mode, NSS doesn't let you import keys. You can either have
> >> keys in a NSS DB, or you can wrap/unwrap them into one (or the
> >> temporary, in-memory key store). But you can't just call
> >> PK11_ImportSymKey (or the related function for private keys) --
> >> they'll fail!
> >
> >> That's because NSS was a FIPS level 2 certified provider, unlike
> >> say, OpenSSL which only undergoes FIPS level 1 certification.
> >> Some of those restrictions that level 2 places are around key
> >> management and handling, including where they're sourced from and
> >> how they're protected in transit.
>
> Gotcha. Yeah, any idiot can "openssl genkey" and do whatever they want
> with the key including publishing it on their web site.
I mean, its not that you _can't_ do that with NSS DBs, its just that
its a lot harder to do, especially if that NSS DB is in FIPS mode.
You'd have to be cognizant of what you're doing. You'd know you're
extracting it (the only way is as a wrapped, password-protected .p12
file) and then using some other tool (potentially on a different system
not in FIPS mode) to unwrap it and publish it.
>
> >> One of the RFEs I've been discussing with our JDK team have been
> >> to loosen the restrictions of the SunJSSE provider in FIPS mode.
> >> Restricting the origin of the KeyManager (while having good
> >> intentions of preventing the above) means that some otherwise
> >> valid use cases -- like selecting a single key alias from the
> >> server.xml config -- won't work. Really they should only care
> >> that the key used is from the SunPKCS11-NSS-FIPS provider, they
> >> don't care how they got it and from which KeyManager it came
> >> from.
>
> Well, if it didn't come from a SunJSSE KeyManager, then it can't prove
> that it wasn't tampered-with -- or even completely replaced, which
> would effectively be an "imported key". So it kind of does make sense.
But you can. SunPKCS11-NSS-FIPS Keys (which a KeyManager just facilitates
access to) are just pointers. You'd need to do a huge amount of unsafe
and conscious work to tamper with that key -- usually using JNI code
and accessing NSS primitives -- in a way that violates any safety
concerns. And you could still do all of that work if you leave it in the
NSS DB and leave their pointer alone (you'd just do it before or after
loading the SunPKCS11-NSS-FIPS key from their KeyManager).
My assertion is that the correct test isn't "is this the right KeyManager
class" but "is this the right _key_ / _certificate_ class"?
The same problem applies to TrustManagers, except in reverse. If you
restrict what TrustManagers are used (IIRC, the SunJSSE + SunPKCS11-NSS-FIPS
does), you can't easily add additional validation. One of our requirements
is to do is full-chain OCSP checking (from CC certification). The default
SunX509 TrustManager doesn't do that. Most people probably don't want that.
Yeah, you can shoot yourself in the foot and write a silly TrustManager.
Anyone can do that, but they'd know what they're doing. And NSS lets you
export public keys, so additional trust managers are possible in theory.
At any rate, these are OpenJDK/SunJSSE problems and not Tomcat problems.
But it does tell you a little of what we have to work with. :-)
~snip older context~
> Can I just say that "KBKDF" is a hilarious concept to me?
>
> Now we just need a KBPDF and we can go full-circle.
:D I don't disagree, but it does have some interesting use cases.
Smart cards have already established trust via symmetric keys when
you initially provisioned them. When they're out in the field, and
you want to put new certificates on them, you can reuse the existing
symmetric keys and create a "secure channel" (SCP 03 is the protocol
name) using KBKDF. It lets you prove integrity of the channel and
identity of the card assuming you trust the initial key and the
hardware to not give up the long-term secret.
It is also used in Kerberos for similar purposes in RFC 8009.
>
> >> I'd like to get to the point where either work but both need a
> >> few bug fixes and/or RFEs first. :-)
> >
> >
> >
> >>>>> However, currently on RHEL the SunJSSE provider is a bit
> >>>>> broken, so this FIPSImplementation isn't really useful. If
> >>>>> you're interested, I'd like to get Tomcat to a place where
> >>>>> it supports FIPS mode JDK without the need for a separate
> >>>>> implementation, but first I need a working FIPS-mode JDK
> >>>>> (currently it doesn't handshake)... ...and we're probably
> >>>>> going the route of finishing JSS anyways.
> >
> > Hmm. Is your goal to get a working JSSE+FIPS implementation
> > specifically with JSS or do you just want to get FIPS working with
> > Tomcat? The OpenSSL provider can be put into FIPS mode, provided
> > that it was build with FIPS support of course.
> >
> >> I think I covered OpenSSL above. We have considered it and it
> >> would be possible. But we'd prefer a NSS backed provider.
> >
> >> So either JSSE+SunPKCS11-NSS-FIPS or JSSE+/-JSS. :-)
>
> Understood.
>
> Mark will probably read this novel and say "oh, that's easy" and just
> implement it. Having not written the original code and not
> understanding the various intricacies of why the replacement
> KeyManager is being used, etc., I wouldn't want to break things for
> large numbers of people.
>
> Have you tried hacking Tomcat to just never wrap the KeyManager and
> use a stock JSSE one? Does it work in that case?
No...
But since you asked and its a small change...
And its Friday... :-)
I made this change since I'm lazy:
diff --git a/java/org/apache/tomcat/util/net/SSLUtilBase.java b/java/org/apache/tomcat/util/net/SSLUtilBase.java
index bcbe3903e8..d8635f757a 100644
--- a/java/org/apache/tomcat/util/net/SSLUtilBase.java
+++ b/java/org/apache/tomcat/util/net/SSLUtilBase.java
@@ -374,9 +374,6 @@ public abstract class SSLUtilBase implements SSLUtil {
if ("JKS".equals(certificate.getCertificateKeystoreType())) {
alias = alias.toLowerCase(Locale.ENGLISH);
}
- for(int i = 0; i < kms.length; i++) {
- kms[i] = new JSSEKeyManager((X509KeyManager)kms[i], alias);
- }
}
return kms;
Obviously that won't merge upstream without more understanding.
This starts without any exceptions (unlike before):
Dec 06, 2019 3:28:37 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["https-jsse-nio-8443"]
Dec 06, 2019 3:28:37 PM org.apache.catalina.startup.Catalina load
INFO: Server initialization in [826] milliseconds
Dec 06, 2019 3:28:37 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service [Catalina]
Dec 06, 2019 3:28:37 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet engine: [Apache Tomcat/9.0.30-dev]
Dec 06, 2019 3:28:37 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["https-jsse-nio-8443"]
Dec 06, 2019 3:28:37 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in [40] milliseconds
And we get to where I got with my TomcatJSSE FIPS mode implementation:
[root@localhost tomcat]# wget https://localhost:8443
--2019-12-06 15:29:13-- https://localhost:8443/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:8443... connected.
GnuTLS: A TLS fatal alert has been received.
GnuTLS: received alert [80]: Internal error
Unable to establish SSL connection.
And in the logs:
Dec 06, 2019 3:29:13 PM org.apache.tomcat.util.net.NioEndpoint$SocketProcessor doRun
SEVERE: Error running socket processor
java.lang.RuntimeException: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ATTRIBUTE_VALUE_INVALID
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1519)
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:802)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at org.apache.tomcat.util.net.SecureNioChannel.handshakeUnwrap(SecureNioChannel.java:499)
at org.apache.tomcat.util.net.SecureNioChannel.handshake(SecureNioChannel.java:238)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1575)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.security.ProviderException: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ATTRIBUTE_VALUE_INVALID
at sun.security.pkcs11.P11KeyPairGenerator.generateKeyPair(P11KeyPairGenerator.java:424)
at java.security.KeyPairGenerator$Delegate.generateKeyPair(KeyPairGenerator.java:697)
at sun.security.ssl.ECDHCrypt.<init>(ECDHCrypt.java:65)
at sun.security.ssl.ServerHandshaker.setupEphemeralECDHKeys(ServerHandshaker.java:1516)
at sun.security.ssl.ServerHandshaker.trySetCipherSuite(ServerHandshaker.java:1311)
at sun.security.ssl.ServerHandshaker.chooseCipherSuite(ServerHandshaker.java:1108)
at sun.security.ssl.ServerHandshaker.clientHello(ServerHandshaker.java:814)
at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:221)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
at org.apache.tomcat.util.net.SecureNioChannel.tasks(SecureNioChannel.java:443)
at org.apache.tomcat.util.net.SecureNioChannel.handshakeUnwrap(SecureNioChannel.java:507)
... 7 more
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ATTRIBUTE_VALUE_INVALID
at sun.security.pkcs11.wrapper.PKCS11.C_GenerateKeyPair(Native Method)
at sun.security.pkcs11.P11KeyPairGenerator.generateKeyPair(P11KeyPairGenerator.java:416)
... 21 more
So clearly the SunJSSE in FIPS mode needs a little bit more work.
(https://bugzilla.redhat.com/show_bug.cgi?id=1780339)
But I was correct about the KeyManager being the issue. \o/
So I think we should find Mark and ask what's the best path
forward.
I'd still like to see if we can get the JDK restrictions loosened
just a little. I have a few downstream trackers I'm watching for that.
But if those fail...
I'm partial to detecting the class of the KeyManager (it'd be
sun.security.ssl.SunX509KeyManagerImpl in the event we're using
JSSE), but we'd need to couple that with another FIPS-mode check...
probably it'd be sufficient to see if the SunPKCS11-NSS-FIPS provider
is present. And check which provide we're using for the SSLEngine.
However... non-PKCS#11 KeyStore's are not allowed in FIPS mode either
(including both JKS and PKCS#12 files, go figure). For maximal usability,
I'd suggest detecting bad configurations and helping users towards fixing
their config. That'd be elsewhere and unrelated to this KeyManager change
though.
Really up to you folks how much you'd like to take upstream. I'm happy
to help and contribute as desired,
- Alex
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat - No Fork for debugging?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Alex,
On 12/6/19 10:57, Alex Scheel wrote:
> Apologies, this reply got away from me... :-)
Not at all. This is very helpful.
> Perhaps grab a fresh $BEVERAGE before reading.
It's before noon, so I grabbed a fresh espresso.
:)
> ----- Original Message -----
>> From: "Christopher Schultz" <ch...@christopherschultz.net> To:
>> users@tomcat.apache.org Sent: Friday, December 6, 2019 9:51:52
>> AM Subject: Re: Tomcat - No Fork for debugging?
>>
> Alex,
>
> On 12/5/19 18:05, Alex Scheel wrote:
>>>> ----- Original Message -----
>>>>> From: "Christopher Schultz" <ch...@christopherschultz.net>
>>>>> To: users@tomcat.apache.org Sent: Thursday, December 5,
>>>>> 2019 5:05:42 PM Subject: Re: Tomcat - No Fork for
>>>>> debugging?
>>>>>
>>>> Alex,
>>>>
>>>> On 12/5/19 15:18, Alex Scheel wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I've tried searching to no avail.
>>>>>>>
>>>>>>> I'm working on a(nother) SSL adapter. However, I've had
>>>>>>> some issues with it. There's a native component and I'm
>>>>>>> trying to tease apart its relationship with why the
>>>>>>> client won't handshake. The stack traces aren't overly
>>>>>>> helpful and I'd love to attach gdb to this and set a
>>>>>>> few specific environment variables.
>>>>>>>
>>>>>>> Query:
>>>>>>>
>>>>>>> Does Tomcat have a mode where it won't fork to a
>>>>>>> different user and will run with a limited number of
>>>>>>> threads? That'd greatly improve my ability to debug.
>>>>>>> Something similar to `radiusd -X` or `sshd -d`?
>>>>
>>>> You can accomplish this in two steps:
>>>>
>>>> 1. Change your <Connector> to have an <Executor> with only a
>>>> single thread, or set maxThreads="1" on the <Connector> if
>>>> you aren't separately configuring an <Executor>
>>>>
>>>> 2. From the command line, run:
>>>>
>>>> $ bin/catalina.sh run
>>>>
>>>>> I think that behaves better. Thanks!
>
> No problem.
>
>>>> This will run Tomcat directly in the terminal window (or, on
>>>> Windows, open a second terminal window where it will run).
>>>>
>>>> You can see stdout in that terminal, and you can press CTRL-C
>>>> to kill the process.
>>>>
>>>>>>> Rationale:
>>>>>>>
>>>>>>> NSS has support for logging calls to its PKCS#11
>>>>>>> interface to a file, based on the presence of
>>>>>>> environment variables. When I set these environment
>>>>>>> variables and directly call the JVM to start Tomcat:
>>>>>>>
>>>>>>> # java -classpath $CLASSPATH $FLAGS
>>>>>>> org.apache.catalina.startup.Bootstrap start
>>>>>>>
>>>>>>> I see it logging calls when the JDK starts up, but when
>>>>>>> I hit it with wget on the TLS port, the resulting
>>>>>>> PKCS#11 calls aren't logged. When launching in gdb, I
>>>>>>> get an error about /sbin/nologin doesn't understand the
>>>>>>> -c option, which to me says that Bootstrap is trying to
>>>>>>> fork and create a new shell (I'm running as root in a
>>>>>>> VM and it wants to launch as the tomcat user), dropping
>>>>>>> my environmental variables I want.
>>>>>>>
>>>>>>> Ideally (for debugging) I'd like to simplify this. Is
>>>>>>> there a more direct entry route I can use perhaps?
>>>>
>>>> Is there a JSSE wrapper for NSS? You can just plug-in the
>>>> crypto provider for the SSLContext instead of writing your
>>>> own connector. Wait. You said "adapter". What kind of
>>>> "adapter" are you writing?
>>>>
>>>>> Ah sorry, my terminology was loose there. I'm writing
>>>>> another org.apache.tomcat.util.net.SSLImplementation
>>>>> implementation. Heh.
>>>>
>>>>> I'm the maintainer of JSS, a NSS wrapper in Java. This is
>>>>> mostly utilized by the Dogtag project.
>>>>
>>>>> I also co-maintain TomcatJSS, a JSS adapter for Tomcat <
>>>>> 8.5. When used with Tomcat >= 8.5, it uses JSSE's SSLEngine
>>>>> but initializes JSS based on configuration in server.xml
>>>>> and use it for the keystore. Mostly this is because we've
>>>>> not yet gotten around to adding a SSLEngine to JSS (but
>>>>> will likely finish that work soon -- JSS was started at
>>>>> Netscape back in the late 90s and predated the SSLEngine
>>>>> being added to JVM interfaces in JDK version 5). RHEL 7
>>>>> still ships with Tomcat 7, but RHEL 8 only has Tomcat 9, so
>>>>> the hard requirement for a SSLEngine is new. We're getting
>>>>> there :)
>>>>
>>>>> With FIPS certification requirements though, we can't use
>>>>> JSSE (since it isn't FIPS certified).
>
> I was pretty sure Java's crypto implementation was FIPS certified.
> I haven't looked in a while, but a quick good search turned up:
>
> https://www2.cs.duke.edu/csed/java/jdk1.7/technotes/guides/security/js
se
>
>
/FIPS.html
>
> and
>
> https://docs.oracle.com/javase/6/docs/technotes/guides/security/enhanc
em
>
>
ents.html
>
> That last one suggests that Oracle Java already contains a JSSE
> provider which uses NSS. DO I misunderstad?
>
>> Also relevant:
>
>> https://docs.oracle.com/javase/8/docs/technotes/guides/security/p11gu
ide.html#NSS
>
>>
>
>
>> Ugh... This is where I confess I'm not an expert on FIPS stuff.
>> Here's my best understanding:
>
>> We're talking about the same provider (that's good!). The subtle
>> point I think you're missing is that FIPS isn't the _default_
>> configuration and it imposes more restrictions when in FIPS mode.
>> It also needs special configuration (which in RHEL looks like):
>
>> fips.provider.1=sun.security.pkcs11.SunPKCS11
>> ${java.home}/lib/security/nss.fips.cfg
>> fips.provider.2=sun.security.provider.Sun
>> fips.provider.3=sun.security.ec.SunEC
>> fips.provider.4=com.sun.net.ssl.internal.ssl.Provider
>> SunPKCS11-NSS-FIPS <--- "FIPS"
>
>> In comparison, the default (non-FIPS mode providers) are:
>
>> security.provider.1=sun.security.provider.Sun
>> security.provider.2=sun.security.rsa.SunRsaSign
>> security.provider.3=sun.security.ec.SunEC
>> security.provider.4=com.sun.net.ssl.internal.ssl.Provider <---
>> "Non-FIPS" security.provider.5=com.sun.crypto.provider.SunJCE
>> security.provider.6=sun.security.jgss.SunProvider
>> security.provider.7=com.sun.security.sasl.Provider
>> security.provider.8=org.jcp.xml.dsig.internal.dom.XMLDSigRI
>> security.provider.9=sun.security.smartcardio.SunPCSC
>> security.provider.10=sun.security.pkcs11.SunPKCS11
>> ${java.home}/lib/security/nss.cfg
>
>> Now, where my understanding gets more fuzzy...
>
>
>> RHEL ships OpenJDK and upstream itself doesn't do the FIPS
>> certification; it is Red Hat that does the certification of NSS
>> on RHEL. JDK itself, under the FIPS configuration above, now does
>> no crypto. All the crypto is done via PKCS#11 to NSS. So there's
>> nothing (in OpenJDK) to certify any more. Unlike in the non-FIPS
>> mode above, there's lots of crypto that would need to be
>> certified.
Aah, okay. This makes more sense to me, now. At this point, the JSSE
provider is just a pass-through to NSS, just like the JSSE wrapper for
OpenSSL.
>> Oracle, in their JDK, might ship their own PKCS#11 provider
>> and/or certify NSS, I don't know. I know IBM ships their own FIPS
>> crypto provider, and probably the next most popular is
>> BouncyCastle, but I thought that might be losing popularity
>> because they might've stopped FIPS certifying it, who knows.
IMHO FIPS is useless at this point unless you are required to use a
FIPS-certified module, which many people are unfortunately required to
use. The reason I think it's useless is because FIPS mandates the use
of certain algorithms and some of them shouldn't be used.
>> But my key point is there's no "OpenJDK FIPS Certificate" on the
>> FIPS page:
>
>> https://csrc.nist.gov/projects/cryptographic-module-validation-progra
m/validated-modules/search
>
>> (Search Java)
>
>> But there is one for NSS ("NSS" - and then its "NSS Cryptographic
>> Module" on the results page).
Yep. I saw that Oracle has FIPS-certified modules, but they aren't a
part of the standard Java distribution (and certainly not part of
OpenJDK). I guess you get what you pay for.
>> So yes, the SunJSSE provider is still used but the key
>> differences are:
>
>> 1) The crypto is done by SunPKCS11-NSS-FIPS provider, 2) The NSS
>> that is used is FIPS certified, 3) It places additional
>> restrictions (more about that below).
>
>> If you don't really care 2 (the token "Certificate") and just
>> want to test stuff, 1 and 3 work on any distro (e.g., Ubuntu and
>> Fedora), and NSS everywhere understands FIPS mode.
>
>> Also the SunPKCS11 / SunPKCS11-NSS-FIPS providers are limited to
>> Linux, iirc.
Well, who would bother with NSS on Windows?
>> That's how I understand the above and no, you don't seem to
>> misunderstand. :)
>
>>>>> OpenJDK supports NSS via the Sun-PKCS11 provider and has a
>>>>> SSLEngine implementation under the SunJSSE provider. It has
>>>>> some restrictions on it that means it can't be used from
>>>>> Tomcat's JSSE adapter just by swapping out the provider
>>>>> name (for one, wrapping the KeyManager like Tomcat does
>>>>> isn't allowed).
>
> Hmm. Maybe there's a better way to do it, then, and you don't have
> to write so much code. What's the problem with wrapping a
> KeyManager?
>
>> That's an RFE I've filed internally against the JDK team. To me,
>> it makes no sense. It fails now with Tomcat's JSSEImplementation
>> :
>
>> Dec 06, 2019 10:20:04 AM org.apache.catalina.util.LifecycleBase
>> handleSubClassException SEVERE: Failed to initialize component
>> [Connector[HTTP/1.1-8443]]
>> org.apache.catalina.LifecycleException: Protocol handler
>> initialization failed at
>> org.apache.catalina.connector.Connector.initInternal(Connector.java:9
35)
>>
>>
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
>> at
>> org.apache.catalina.core.StandardService.initInternal(StandardService
.java:530)
>>
>>
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
>> at
>> org.apache.catalina.core.StandardServer.initInternal(StandardServer.j
ava:852)
>>
>>
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
>> at org.apache.catalina.startup.Catalina.load(Catalina.java:633)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:62)
>>
>>
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498) at
>> org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:306) at
>> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:491)
>> Caused by: java.lang.IllegalArgumentException: FIPS mode: only
>> SunJSSE KeyManagers may be used at
>> org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(Abst
ractJsseEndpoint.java:114)
>>
>>
at
org.apache.tomcat.util.net.AbstractJsseEndpoint.initialiseSsl(AbstractJs
seEndpoint.java:85)
>> at
>> org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:216)
>>
>>
at
org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:1
043)
>> at
>> org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:540)
>>
>>
at
org.apache.coyote.http11.AbstractHttp11Protocol.init(AbstractHttp11Proto
col.java:74)
>> at
>> org.apache.catalina.connector.Connector.initInternal(Connector.java:9
32)
>>
>>
... 12 more
>> Caused by: java.security.KeyManagementException: FIPS mode: only
>> SunJSSE KeyManagers may be used at
>> sun.security.ssl.SSLContextImpl.chooseKeyManager(SSLContextImpl.java:
154)
>>
>>
at sun.security.ssl.SSLContextImpl.engineInit(SSLContextImpl.java:71)
>> at javax.net.ssl.SSLContext.init(SSLContext.java:282) at
>> org.apache.tomcat.util.net.jsse.JSSESSLContext.init(JSSESSLContext.ja
va:53)
>>
>>
at
org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(Abstrac
tJsseEndpoint.java:112)
>> ... 18 more
>
Aha. "What's wrong with wrapping the KeyManager?"
Answer: it doesn't work: KeyManagementException
I guess they don't want any funny business.
I'll have to look at how the KeyManager is built/wrapped. I didn't
write that code so I'm not too familiar with it. I know we need to
wave our hands around certain things due to the flexibility Tomcat
allows for certificate-selection, etc.
When you get that error, is the object of type
org/apache/tomcat/util/net/jsse/JSSEKeyManager?
I wonder if we are jumping the gun and always using that KeyManager
even when it's not strictly necessary.
>> My config roughly looks like:
>
>> <Connector name="Secure" port="8443" scheme="https" secure="true"
>> SSLEnabled="true" keyAlias="localhost"
>> keystorePass="keystorePass" keystoreType="PKCS11"
>> keystoreProvider="SunPKCS11-NSS-FIPS"
>> keyManagerAlgorithm="SunX509" truststoreAlgorithm="PKCS11"
>> truststoreProvider="SunJSSE" truststoreType="SunX509"
>> truststorePassword="truststorePassword" SSLProtocol="TLSv1.2"
>> maxThreads="1" />
>
>> (Yeah, that's messy -- but it illustrates the point -- you can
>> probably remove a number of those elements -- *including*
>> keyAlias, and it'll still fail).
>
>> This is because, for all KeyManagers, Tomcat wraps it its own
>> KeyManager:
>
>> https://github.com/apache/tomcat/blob/master/java/org/apache/tomcat/u
til/net/SSLUtilBase.java#L378
>
>> Probably the minimum untested change that I'd have proposed
>> would be to wrap it in an if block like you have ealier:
>
>> https://github.com/apache/tomcat/blob/master/java/org/apache/tomcat/u
til/net/SSLUtilBase.java#L195
Sounds
>>
like the same as my conclusion. I wouldn't want to make that
change myself without understanding the other implications.
>> Now the core of the _why_ is because that's how NSS behaves. In
>> FIPS mode, NSS doesn't let you import keys. You can either have
>> keys in a NSS DB, or you can wrap/unwrap them into one (or the
>> temporary, in-memory key store). But you can't just call
>> PK11_ImportSymKey (or the related function for private keys) --
>> they'll fail!
>
>> That's because NSS was a FIPS level 2 certified provider, unlike
>> say, OpenSSL which only undergoes FIPS level 1 certification.
>> Some of those restrictions that level 2 places are around key
>> management and handling, including where they're sourced from and
>> how they're protected in transit.
Gotcha. Yeah, any idiot can "openssl genkey" and do whatever they want
with the key including publishing it on their web site.
>> One of the RFEs I've been discussing with our JDK team have been
>> to loosen the restrictions of the SunJSSE provider in FIPS mode.
>> Restricting the origin of the KeyManager (while having good
>> intentions of preventing the above) means that some otherwise
>> valid use cases -- like selecting a single key alias from the
>> server.xml config -- won't work. Really they should only care
>> that the key used is from the SunPKCS11-NSS-FIPS provider, they
>> don't care how they got it and from which KeyManager it came
>> from.
Well, if it didn't come from a SunJSSE KeyManager, then it can't prove
that it wasn't tampered-with -- or even completely replaced, which
would effectively be an "imported key". So it kind of does make sense.
>> Those changes are still under discussion.
>
>> ---
>
>> There's also about ~4 or 5 other weird bugs in this SunJSSE FIPS
>> interface that make it less than usable or cause it to fail to
>> handshake in FIPS mode, but they don't really concern Tomcat.
>
>> Those are why I'm not proposing a PR yet. :-)
>
>
>>>>> I was trying to see if the JDK's SSLEngine would work for
>>>>> us. On RHEL, we FIPS certify our shipped NSS release so the
>>>>> JDK's Sun-PKCS11-NSS-FIPS provider is FIPS certified
>>>>> transitively (since it does no crypto... :). Since I'm not
>>>>> a Tomcat developer (and am more familiar with TomcatJSS), I
>>>>> stubbed out a new SSLImplementation that uses SunJSSE so I
>>>>> could bypass the parts of the Tomcat JSSEImplementation
>>>>> that don't work with the JDK SunJSSE provider.
>
> That sounds like it would work, but also sound like a huge amount
> of plumbing. What can you tell me about the way Tomcat uses the
> JSSE API that precludes FIPS support from the existing tools --
> including your own JSS?
>
>> Mostly that's discussed above w.r.t. SunJSSE +
>> SunPKCS11-NSS-FIPS.
>
>
>> Why JSS?
>
>> Dogtag is a self-hosted CA using NSS and came out of Netscape's
>> certificate offerings before landing at Red Hat. We support RSA
>> and ECDSA certificates and also have strong HSM support (thanks
>> to NSS). In part, this means we can install a CA and have both
>> the CA and the SSL server certificates for the CA interface never
>> leave the HSM (or the NSS DB).
"We are a CA" is basically all the explanation I needed :)
>> Since (especially in FIPS mode) HSMs generally frown upon
>> extracting keys, that makes using say, BouncyCastle or Tomcat's
>> OpenSSL adapter harder, since we'd need to configure and support
>> HSM access from both NSS and $PROVIDER simultaneously. Far easier
>> to use a PKCS#11 provider with NSS (where we've already set up
>> the HSM and NSS DB). That leaves well, JSS or
>> SunJSSE+SunPKCS11-NSS.
>
>> For internal reasons, the FIPS mode SunJSSE provider wasn't
>> well... supported until relatively recently. So for most of our
>> time, we've been fine using JSS (via TomcatJSS's BIO adapter on
>> Tomcat < 8.5). Its only recently that we've needed an NIO adapter
>> (so JSS hasn't yet gotten a SSLEngine) -- and only recently that
>> we've considered SunJSSE again.
>
>
>> So:
>
>> 1) We can't use Tomcat's JSSE API because we don't have a
>> SSLEngine in JSS yet, 2) We can't use SunJSSE+SunPKCS11-NSS-FIPS
>> because it doesn't work.
>
>> Currently we ignore FIPS (on Tomcat >= 8.5) and use the regular,
>> non-FIPS SunJSSE. The only thing we really use TomcatJSS for is
>> to initialize JSS early enough in the Tomcat startup sequence
>> (before the app starts up) -- so that we can use JSS as the
>> keystore for Tomcat and SunJSSE (which the non-FIPS mode adapter
>> is happy to do).
>
>> For the rest of the app, we do need JSS though. In part because
>> its one of the few Java providers that provide CMAC and (soon)
>> KBKDF, but we also use it for all our certificate issuance
>> operations.
Can I just say that "KBKDF" is a hilarious concept to me?
Now we just need a KBPDF and we can go full-circle.
>> I'd like to get to the point where either work but both need a
>> few bug fixes and/or RFEs first. :-)
>
>
>
>>>>> However, currently on RHEL the SunJSSE provider is a bit
>>>>> broken, so this FIPSImplementation isn't really useful. If
>>>>> you're interested, I'd like to get Tomcat to a place where
>>>>> it supports FIPS mode JDK without the need for a separate
>>>>> implementation, but first I need a working FIPS-mode JDK
>>>>> (currently it doesn't handshake)... ...and we're probably
>>>>> going the route of finishing JSS anyways.
>
> Hmm. Is your goal to get a working JSSE+FIPS implementation
> specifically with JSS or do you just want to get FIPS working with
> Tomcat? The OpenSSL provider can be put into FIPS mode, provided
> that it was build with FIPS support of course.
>
>> I think I covered OpenSSL above. We have considered it and it
>> would be possible. But we'd prefer a NSS backed provider.
>
>> So either JSSE+SunPKCS11-NSS-FIPS or JSSE+/-JSS. :-)
Understood.
Mark will probably read this novel and say "oh, that's easy" and just
implement it. Having not written the original code and not
understanding the various intricacies of why the replacement
KeyManager is being used, etc., I wouldn't want to break things for
large numbers of people.
Have you tried hacking Tomcat to just never wrap the KeyManager and
use a stock JSSE one? Does it work in that case?
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl3qksMACgkQHPApP6U8
pFj8bg/8C50dYqQIv7mpvJ3ntdChjfn2FQdcNrYqpbOSrREaXaeSHShGAoHyjDnN
sP8UbVQ8jyie/RC1a8l2wKPozFMv21n6p8q6USU8N0g0PC2oYidJC+FqQx0MwSto
oqrXYxAR+wcY/HZXmFH56xOG2Lbw9AcBO7NUgaRGFapoYvYXiAAh9x61rRrTXjOX
3enJx3L7r40F4b8hug8o42S87TMonWoURhBoIx1grAx29jq8eBymy/Hp+LuyD5GH
PYZ3ICgCqg2vSqMeuYRbt+WswwPiWMfuFKcj5Kd7F63sQUValhEmLMPs7Rj/ZukI
Ow5d7Han8nmSTtslzCIOCSOGj2eVTcqZOlGW9fs9tpSfKFVbdKFzFm7uGSQApv0j
h8l08RK8kMBS567K8VGl5VHD9vGiTxfqME7e5SzRmrS/PfYnAAF40dmbulJbGHKV
V7HupYASkIzRj2/+yht3h0auZUr/qe86LBGNUbwzSus9Mp++VfKNLmNxU26C6VlU
G4SUQcPIIzgjmcSsW/asR3EvbjPdQoTAcWacFcXvWNqsiu9D3Aotb0VwQR8OnPSL
dMQrXuHS+ynsc8Ywe40+x/g1i+HOSJPmfSYOI2QYCPdF4yMz/sNDdNQpMuvnlbot
mvgPczhTRBy8pJiPwixwxuy93YmkblrGkJV9fUPfkLi913cPqIA=
=HJ2y
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat - No Fork for debugging?
Posted by Alex Scheel <as...@redhat.com>.
Apologies, this reply got away from me... :-)
Perhaps grab a fresh $BEVERAGE before reading.
----- Original Message -----
> From: "Christopher Schultz" <ch...@christopherschultz.net>
> To: users@tomcat.apache.org
> Sent: Friday, December 6, 2019 9:51:52 AM
> Subject: Re: Tomcat - No Fork for debugging?
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Alex,
>
> On 12/5/19 18:05, Alex Scheel wrote:
> > ----- Original Message -----
> >> From: "Christopher Schultz" <ch...@christopherschultz.net> To:
> >> users@tomcat.apache.org Sent: Thursday, December 5, 2019 5:05:42
> >> PM Subject: Re: Tomcat - No Fork for debugging?
> >>
> > Alex,
> >
> > On 12/5/19 15:18, Alex Scheel wrote:
> >>>> Hi all,
> >>>>
> >>>> I've tried searching to no avail.
> >>>>
> >>>> I'm working on a(nother) SSL adapter. However, I've had some
> >>>> issues with it. There's a native component and I'm trying to
> >>>> tease apart its relationship with why the client won't
> >>>> handshake. The stack traces aren't overly helpful and I'd
> >>>> love to attach gdb to this and set a few specific environment
> >>>> variables.
> >>>>
> >>>> Query:
> >>>>
> >>>> Does Tomcat have a mode where it won't fork to a different
> >>>> user and will run with a limited number of threads? That'd
> >>>> greatly improve my ability to debug. Something similar to
> >>>> `radiusd -X` or `sshd -d`?
> >
> > You can accomplish this in two steps:
> >
> > 1. Change your <Connector> to have an <Executor> with only a
> > single thread, or set maxThreads="1" on the <Connector> if you
> > aren't separately configuring an <Executor>
> >
> > 2. From the command line, run:
> >
> > $ bin/catalina.sh run
> >
> >> I think that behaves better. Thanks!
>
> No problem.
>
> > This will run Tomcat directly in the terminal window (or, on
> > Windows, open a second terminal window where it will run).
> >
> > You can see stdout in that terminal, and you can press CTRL-C to
> > kill the process.
> >
> >>>> Rationale:
> >>>>
> >>>> NSS has support for logging calls to its PKCS#11 interface to
> >>>> a file, based on the presence of environment variables. When
> >>>> I set these environment variables and directly call the JVM
> >>>> to start Tomcat:
> >>>>
> >>>> # java -classpath $CLASSPATH $FLAGS
> >>>> org.apache.catalina.startup.Bootstrap start
> >>>>
> >>>> I see it logging calls when the JDK starts up, but when I hit
> >>>> it with wget on the TLS port, the resulting PKCS#11 calls
> >>>> aren't logged. When launching in gdb, I get an error about
> >>>> /sbin/nologin doesn't understand the -c option, which to me
> >>>> says that Bootstrap is trying to fork and create a new shell
> >>>> (I'm running as root in a VM and it wants to launch as the
> >>>> tomcat user), dropping my environmental variables I want.
> >>>>
> >>>> Ideally (for debugging) I'd like to simplify this. Is there a
> >>>> more direct entry route I can use perhaps?
> >
> > Is there a JSSE wrapper for NSS? You can just plug-in the crypto
> > provider for the SSLContext instead of writing your own connector.
> > Wait. You said "adapter". What kind of "adapter" are you writing?
> >
> >> Ah sorry, my terminology was loose there. I'm writing another
> >> org.apache.tomcat.util.net.SSLImplementation implementation.
> >> Heh.
> >
> >> I'm the maintainer of JSS, a NSS wrapper in Java. This is mostly
> >> utilized by the Dogtag project.
> >
> >> I also co-maintain TomcatJSS, a JSS adapter for Tomcat < 8.5.
> >> When used with Tomcat >= 8.5, it uses JSSE's SSLEngine but
> >> initializes JSS based on configuration in server.xml and use it
> >> for the keystore. Mostly this is because we've not yet gotten
> >> around to adding a SSLEngine to JSS (but will likely finish that
> >> work soon -- JSS was started at Netscape back in the late 90s and
> >> predated the SSLEngine being added to JVM interfaces in JDK
> >> version 5). RHEL 7 still ships with Tomcat 7, but RHEL 8 only
> >> has Tomcat 9, so the hard requirement for a SSLEngine is new.
> >> We're getting there :)
> >
> >> With FIPS certification requirements though, we can't use JSSE
> >> (since it isn't FIPS certified).
>
> I was pretty sure Java's crypto implementation was FIPS certified. I
> haven't looked in a while, but a quick good search turned up:
>
> https://www2.cs.duke.edu/csed/java/jdk1.7/technotes/guides/security/jsse
> /FIPS.html
>
> and
>
> https://docs.oracle.com/javase/6/docs/technotes/guides/security/enhancem
> ents.html
>
> That last one suggests that Oracle Java already contains a JSSE
> provider which uses NSS. DO I misunderstad?
Also relevant:
https://docs.oracle.com/javase/8/docs/technotes/guides/security/p11guide.html#NSS
Ugh... This is where I confess I'm not an expert on FIPS stuff. Here's my
best understanding:
We're talking about the same provider (that's good!). The subtle point
I think you're missing is that FIPS isn't the _default_ configuration and
it imposes more restrictions when in FIPS mode. It also needs special
configuration (which in RHEL looks like):
fips.provider.1=sun.security.pkcs11.SunPKCS11 ${java.home}/lib/security/nss.fips.cfg
fips.provider.2=sun.security.provider.Sun
fips.provider.3=sun.security.ec.SunEC
fips.provider.4=com.sun.net.ssl.internal.ssl.Provider SunPKCS11-NSS-FIPS <--- "FIPS"
In comparison, the default (non-FIPS mode providers) are:
security.provider.1=sun.security.provider.Sun
security.provider.2=sun.security.rsa.SunRsaSign
security.provider.3=sun.security.ec.SunEC
security.provider.4=com.sun.net.ssl.internal.ssl.Provider <--- "Non-FIPS"
security.provider.5=com.sun.crypto.provider.SunJCE
security.provider.6=sun.security.jgss.SunProvider
security.provider.7=com.sun.security.sasl.Provider
security.provider.8=org.jcp.xml.dsig.internal.dom.XMLDSigRI
security.provider.9=sun.security.smartcardio.SunPCSC
security.provider.10=sun.security.pkcs11.SunPKCS11 ${java.home}/lib/security/nss.cfg
Now, where my understanding gets more fuzzy...
RHEL ships OpenJDK and upstream itself doesn't do the FIPS certification;
it is Red Hat that does the certification of NSS on RHEL. JDK itself, under
the FIPS configuration above, now does no crypto. All the crypto is done
via PKCS#11 to NSS. So there's nothing (in OpenJDK) to certify any more.
Unlike in the non-FIPS mode above, there's lots of crypto that would need
to be certified.
Oracle, in their JDK, might ship their own PKCS#11 provider and/or certify
NSS, I don't know. I know IBM ships their own FIPS crypto provider, and
probably the next most popular is BouncyCastle, but I thought that might be
losing popularity because they might've stopped FIPS certifying it, who knows.
But my key point is there's no "OpenJDK FIPS Certificate" on the FIPS page:
https://csrc.nist.gov/projects/cryptographic-module-validation-program/validated-modules/search
(Search Java)
But there is one for NSS ("NSS" - and then its "NSS Cryptographic Module"
on the results page).
So yes, the SunJSSE provider is still used but the key differences are:
1) The crypto is done by SunPKCS11-NSS-FIPS provider,
2) The NSS that is used is FIPS certified,
3) It places additional restrictions (more about that below).
If you don't really care 2 (the token "Certificate") and just want to
test stuff, 1 and 3 work on any distro (e.g., Ubuntu and Fedora), and
NSS everywhere understands FIPS mode.
Also the SunPKCS11 / SunPKCS11-NSS-FIPS providers are limited to Linux,
iirc.
That's how I understand the above and no, you don't seem to
misunderstand. :)
> >> OpenJDK supports NSS via the Sun-PKCS11 provider and has a
> >> SSLEngine implementation under the SunJSSE provider. It has some
> >> restrictions on it that means it can't be used from Tomcat's
> >> JSSE adapter just by swapping out the provider name (for one,
> >> wrapping the KeyManager like Tomcat does isn't allowed).
>
> Hmm. Maybe there's a better way to do it, then, and you don't have to
> write so much code. What's the problem with wrapping a KeyManager?
That's an RFE I've filed internally against the JDK team. To me, it
makes no sense. It fails now with Tomcat's JSSEImplementation :
Dec 06, 2019 10:20:04 AM org.apache.catalina.util.LifecycleBase handleSubClassException
SEVERE: Failed to initialize component [Connector[HTTP/1.1-8443]]
org.apache.catalina.LifecycleException: Protocol handler initialization failed
at org.apache.catalina.connector.Connector.initInternal(Connector.java:935)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
at org.apache.catalina.core.StandardService.initInternal(StandardService.java:530)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:852)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
at org.apache.catalina.startup.Catalina.load(Catalina.java:633)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:306)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:491)
Caused by: java.lang.IllegalArgumentException: FIPS mode: only SunJSSE KeyManagers may be used
at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:114)
at org.apache.tomcat.util.net.AbstractJsseEndpoint.initialiseSsl(AbstractJsseEndpoint.java:85)
at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:216)
at org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:1043)
at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:540)
at org.apache.coyote.http11.AbstractHttp11Protocol.init(AbstractHttp11Protocol.java:74)
at org.apache.catalina.connector.Connector.initInternal(Connector.java:932)
... 12 more
Caused by: java.security.KeyManagementException: FIPS mode: only SunJSSE KeyManagers may be used
at sun.security.ssl.SSLContextImpl.chooseKeyManager(SSLContextImpl.java:154)
at sun.security.ssl.SSLContextImpl.engineInit(SSLContextImpl.java:71)
at javax.net.ssl.SSLContext.init(SSLContext.java:282)
at org.apache.tomcat.util.net.jsse.JSSESSLContext.init(JSSESSLContext.java:53)
at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:112)
... 18 more
My config roughly looks like:
<Connector name="Secure" port="8443" scheme="https" secure="true" SSLEnabled="true"
keyAlias="localhost" keystorePass="keystorePass"
keystoreType="PKCS11" keystoreProvider="SunPKCS11-NSS-FIPS"
keyManagerAlgorithm="SunX509" truststoreAlgorithm="PKCS11"
truststoreProvider="SunJSSE" truststoreType="SunX509"
truststorePassword="truststorePassword"
SSLProtocol="TLSv1.2" maxThreads="1" />
(Yeah, that's messy -- but it illustrates the point -- you can probably
remove a number of those elements -- *including* keyAlias, and it'll still
fail).
This is because, for all KeyManagers, Tomcat wraps it its own KeyManager:
https://github.com/apache/tomcat/blob/master/java/org/apache/tomcat/util/net/SSLUtilBase.java#L378
Probably the minimum untested change that I'd have proposed would
be to wrap it in an if block like you have ealier:
https://github.com/apache/tomcat/blob/master/java/org/apache/tomcat/util/net/SSLUtilBase.java#L195
---
Now the core of the _why_ is because that's how NSS behaves. In FIPS
mode, NSS doesn't let you import keys. You can either have keys in a
NSS DB, or you can wrap/unwrap them into one (or the temporary, in-memory
key store). But you can't just call PK11_ImportSymKey (or the related
function for private keys) -- they'll fail!
That's because NSS was a FIPS level 2 certified provider, unlike say,
OpenSSL which only undergoes FIPS level 1 certification. Some of those
restrictions that level 2 places are around key management and handling,
including where they're sourced from and how they're protected in transit.
One of the RFEs I've been discussing with our JDK team have been to
loosen the restrictions of the SunJSSE provider in FIPS mode. Restricting
the origin of the KeyManager (while having good intentions of preventing
the above) means that some otherwise valid use cases -- like selecting a
single key alias from the server.xml config -- won't work. Really they
should only care that the key used is from the SunPKCS11-NSS-FIPS
provider, they don't care how they got it and from which KeyManager it
came from.
Those changes are still under discussion.
---
There's also about ~4 or 5 other weird bugs in this SunJSSE FIPS interface
that make it less than usable or cause it to fail to handshake in FIPS mode,
but they don't really concern Tomcat.
Those are why I'm not proposing a PR yet. :-)
>
> >> I was trying to see if the JDK's SSLEngine would work for us. On
> >> RHEL, we FIPS certify our shipped NSS release so the JDK's
> >> Sun-PKCS11-NSS-FIPS provider is FIPS certified transitively
> >> (since it does no crypto... :). Since I'm not a Tomcat developer
> >> (and am more familiar with TomcatJSS), I stubbed out a new
> >> SSLImplementation that uses SunJSSE so I could bypass the parts
> >> of the Tomcat JSSEImplementation that don't work with the JDK
> >> SunJSSE provider.
>
> That sounds like it would work, but also sound like a huge amount of
> plumbing. What can you tell me about the way Tomcat uses the JSSE API
> that precludes FIPS support from the existing tools -- including your
> own JSS?
Mostly that's discussed above w.r.t. SunJSSE + SunPKCS11-NSS-FIPS.
Why JSS?
Dogtag is a self-hosted CA using NSS and came out of Netscape's
certificate offerings before landing at Red Hat. We support RSA and ECDSA
certificates and also have strong HSM support (thanks to NSS). In part,
this means we can install a CA and have both the CA and the SSL server
certificates for the CA interface never leave the HSM (or the NSS DB).
Since (especially in FIPS mode) HSMs generally frown upon extracting
keys, that makes using say, BouncyCastle or Tomcat's OpenSSL adapter
harder, since we'd need to configure and support HSM access from both
NSS and $PROVIDER simultaneously. Far easier to use a PKCS#11 provider
with NSS (where we've already set up the HSM and NSS DB). That leaves
well, JSS or SunJSSE+SunPKCS11-NSS.
For internal reasons, the FIPS mode SunJSSE provider wasn't well...
supported until relatively recently. So for most of our time, we've
been fine using JSS (via TomcatJSS's BIO adapter on Tomcat < 8.5). Its
only recently that we've needed an NIO adapter (so JSS hasn't yet gotten
a SSLEngine) -- and only recently that we've considered SunJSSE again.
So:
1) We can't use Tomcat's JSSE API because we don't have a SSLEngine in
JSS yet,
2) We can't use SunJSSE+SunPKCS11-NSS-FIPS because it doesn't work.
Currently we ignore FIPS (on Tomcat >= 8.5) and use the regular,
non-FIPS SunJSSE. The only thing we really use TomcatJSS for is to
initialize JSS early enough in the Tomcat startup sequence (before
the app starts up) -- so that we can use JSS as the keystore for
Tomcat and SunJSSE (which the non-FIPS mode adapter is happy to
do).
For the rest of the app, we do need JSS though. In part because its
one of the few Java providers that provide CMAC and (soon) KBKDF,
but we also use it for all our certificate issuance operations.
I'd like to get to the point where either work but both need a few
bug fixes and/or RFEs first. :-)
>
>
> >> However, currently on RHEL the SunJSSE provider is a bit broken,
> >> so this FIPSImplementation isn't really useful. If you're
> >> interested, I'd like to get Tomcat to a place where it supports
> >> FIPS mode JDK without the need for a separate implementation, but
> >> first I need a working FIPS-mode JDK (currently it doesn't
> >> handshake)... ...and we're probably going the route of finishing
> >> JSS anyways.
>
> Hmm. Is your goal to get a working JSSE+FIPS implementation
> specifically with JSS or do you just want to get FIPS working with
> Tomcat? The OpenSSL provider can be put into FIPS mode, provided that
> it was build with FIPS support of course.
I think I covered OpenSSL above. We have considered it and it would
be possible. But we'd prefer a NSS backed provider.
So either JSSE+SunPKCS11-NSS-FIPS or JSSE+/-JSS. :-)
- Alex
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl3qawcACgkQHPApP6U8
> pFg1RBAAi1ANc0Bcf9rlZ/QnjWe4QAg3mA/lftAMjI6/l2QIwbIDf7Pt66a/0e2H
> h/Vzv8xK15+HWovknzrvZrIbHX7duisLeYNV4nV6QCjYeCviiw8ZPTKvrOUcdnfC
> pvYDyZ+K/lTr2dCSVV94trbzYkexgS5cPS6T4zdZuhszMH8Z6dY/lUj6kl4R9eQt
> 6meWPzlXljCOxPamJC2XsgM1dmxwz4MCoI46sj6qPvqNPrpglKEZqPytMl3EXZT1
> UmEYTglYZLixZL9D56WSzF89FyGKKWyYueqRfaI/iM04tupn7baT+k8SVDroRM4g
> I7dkIlJUO/t0WpxIEPHRul7dW+R4I/Dr6miAkm/vhKtVMV5n0gsQL5md7G/eOTls
> 9QJcBZktkzdtvznyBNniMZ3+QFFXZkftuOzB+l3VUwqJ6K0OvtFxMqiy7ET/ZvVA
> ZOyGKnrmIF+JXne+hOAm5Dj80R1DMQl8oITSY3mc/bRPWo96mdfFlc9SYs8ZiDDG
> uaxocXWA8zKRrQNQdVkmHmlB2Tr6i4Nz4MulXlhz7FHLXD2J30mrJuaBBYdmS8qk
> XTlkhflpOsr9hB7GoMkjWcTNjm+mZtcvpLR9p1VJgd5ewounl4d16Az6IphtSyqV
> wXU3KDJ889TSTLlYJoK/s/FCCFsSCM6E3ER9rVbwJsUXEu7C0ow=
> =pu3l
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat - No Fork for debugging?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Alex,
On 12/5/19 18:05, Alex Scheel wrote:
> ----- Original Message -----
>> From: "Christopher Schultz" <ch...@christopherschultz.net> To:
>> users@tomcat.apache.org Sent: Thursday, December 5, 2019 5:05:42
>> PM Subject: Re: Tomcat - No Fork for debugging?
>>
> Alex,
>
> On 12/5/19 15:18, Alex Scheel wrote:
>>>> Hi all,
>>>>
>>>> I've tried searching to no avail.
>>>>
>>>> I'm working on a(nother) SSL adapter. However, I've had some
>>>> issues with it. There's a native component and I'm trying to
>>>> tease apart its relationship with why the client won't
>>>> handshake. The stack traces aren't overly helpful and I'd
>>>> love to attach gdb to this and set a few specific environment
>>>> variables.
>>>>
>>>> Query:
>>>>
>>>> Does Tomcat have a mode where it won't fork to a different
>>>> user and will run with a limited number of threads? That'd
>>>> greatly improve my ability to debug. Something similar to
>>>> `radiusd -X` or `sshd -d`?
>
> You can accomplish this in two steps:
>
> 1. Change your <Connector> to have an <Executor> with only a
> single thread, or set maxThreads="1" on the <Connector> if you
> aren't separately configuring an <Executor>
>
> 2. From the command line, run:
>
> $ bin/catalina.sh run
>
>> I think that behaves better. Thanks!
No problem.
> This will run Tomcat directly in the terminal window (or, on
> Windows, open a second terminal window where it will run).
>
> You can see stdout in that terminal, and you can press CTRL-C to
> kill the process.
>
>>>> Rationale:
>>>>
>>>> NSS has support for logging calls to its PKCS#11 interface to
>>>> a file, based on the presence of environment variables. When
>>>> I set these environment variables and directly call the JVM
>>>> to start Tomcat:
>>>>
>>>> # java -classpath $CLASSPATH $FLAGS
>>>> org.apache.catalina.startup.Bootstrap start
>>>>
>>>> I see it logging calls when the JDK starts up, but when I hit
>>>> it with wget on the TLS port, the resulting PKCS#11 calls
>>>> aren't logged. When launching in gdb, I get an error about
>>>> /sbin/nologin doesn't understand the -c option, which to me
>>>> says that Bootstrap is trying to fork and create a new shell
>>>> (I'm running as root in a VM and it wants to launch as the
>>>> tomcat user), dropping my environmental variables I want.
>>>>
>>>> Ideally (for debugging) I'd like to simplify this. Is there a
>>>> more direct entry route I can use perhaps?
>
> Is there a JSSE wrapper for NSS? You can just plug-in the crypto
> provider for the SSLContext instead of writing your own connector.
> Wait. You said "adapter". What kind of "adapter" are you writing?
>
>> Ah sorry, my terminology was loose there. I'm writing another
>> org.apache.tomcat.util.net.SSLImplementation implementation.
>> Heh.
>
>> I'm the maintainer of JSS, a NSS wrapper in Java. This is mostly
>> utilized by the Dogtag project.
>
>> I also co-maintain TomcatJSS, a JSS adapter for Tomcat < 8.5.
>> When used with Tomcat >= 8.5, it uses JSSE's SSLEngine but
>> initializes JSS based on configuration in server.xml and use it
>> for the keystore. Mostly this is because we've not yet gotten
>> around to adding a SSLEngine to JSS (but will likely finish that
>> work soon -- JSS was started at Netscape back in the late 90s and
>> predated the SSLEngine being added to JVM interfaces in JDK
>> version 5). RHEL 7 still ships with Tomcat 7, but RHEL 8 only
>> has Tomcat 9, so the hard requirement for a SSLEngine is new.
>> We're getting there :)
>
>> With FIPS certification requirements though, we can't use JSSE
>> (since it isn't FIPS certified).
I was pretty sure Java's crypto implementation was FIPS certified. I
haven't looked in a while, but a quick good search turned up:
https://www2.cs.duke.edu/csed/java/jdk1.7/technotes/guides/security/jsse
/FIPS.html
and
https://docs.oracle.com/javase/6/docs/technotes/guides/security/enhancem
ents.html
That last one suggests that Oracle Java already contains a JSSE
provider which uses NSS. DO I misunderstad?
>> OpenJDK supports NSS via the Sun-PKCS11 provider and has a
>> SSLEngine implementation under the SunJSSE provider. It has some
>> restrictions on it that means it can't be used from Tomcat's
>> JSSE adapter just by swapping out the provider name (for one,
>> wrapping the KeyManager like Tomcat does isn't allowed).
Hmm. Maybe there's a better way to do it, then, and you don't have to
write so much code. What's the problem with wrapping a KeyManager?
>> I was trying to see if the JDK's SSLEngine would work for us. On
>> RHEL, we FIPS certify our shipped NSS release so the JDK's
>> Sun-PKCS11-NSS-FIPS provider is FIPS certified transitively
>> (since it does no crypto... :). Since I'm not a Tomcat developer
>> (and am more familiar with TomcatJSS), I stubbed out a new
>> SSLImplementation that uses SunJSSE so I could bypass the parts
>> of the Tomcat JSSEImplementation that don't work with the JDK
>> SunJSSE provider.
That sounds like it would work, but also sound like a huge amount of
plumbing. What can you tell me about the way Tomcat uses the JSSE API
that precludes FIPS support from the existing tools -- including your
own JSS?
>> However, currently on RHEL the SunJSSE provider is a bit broken,
>> so this FIPSImplementation isn't really useful. If you're
>> interested, I'd like to get Tomcat to a place where it supports
>> FIPS mode JDK without the need for a separate implementation, but
>> first I need a working FIPS-mode JDK (currently it doesn't
>> handshake)... ...and we're probably going the route of finishing
>> JSS anyways.
Hmm. Is your goal to get a working JSSE+FIPS implementation
specifically with JSS or do you just want to get FIPS working with
Tomcat? The OpenSSL provider can be put into FIPS mode, provided that
it was build with FIPS support of course.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl3qawcACgkQHPApP6U8
pFg1RBAAi1ANc0Bcf9rlZ/QnjWe4QAg3mA/lftAMjI6/l2QIwbIDf7Pt66a/0e2H
h/Vzv8xK15+HWovknzrvZrIbHX7duisLeYNV4nV6QCjYeCviiw8ZPTKvrOUcdnfC
pvYDyZ+K/lTr2dCSVV94trbzYkexgS5cPS6T4zdZuhszMH8Z6dY/lUj6kl4R9eQt
6meWPzlXljCOxPamJC2XsgM1dmxwz4MCoI46sj6qPvqNPrpglKEZqPytMl3EXZT1
UmEYTglYZLixZL9D56WSzF89FyGKKWyYueqRfaI/iM04tupn7baT+k8SVDroRM4g
I7dkIlJUO/t0WpxIEPHRul7dW+R4I/Dr6miAkm/vhKtVMV5n0gsQL5md7G/eOTls
9QJcBZktkzdtvznyBNniMZ3+QFFXZkftuOzB+l3VUwqJ6K0OvtFxMqiy7ET/ZvVA
ZOyGKnrmIF+JXne+hOAm5Dj80R1DMQl8oITSY3mc/bRPWo96mdfFlc9SYs8ZiDDG
uaxocXWA8zKRrQNQdVkmHmlB2Tr6i4Nz4MulXlhz7FHLXD2J30mrJuaBBYdmS8qk
XTlkhflpOsr9hB7GoMkjWcTNjm+mZtcvpLR9p1VJgd5ewounl4d16Az6IphtSyqV
wXU3KDJ889TSTLlYJoK/s/FCCFsSCM6E3ER9rVbwJsUXEu7C0ow=
=pu3l
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat - No Fork for debugging?
Posted by Alex Scheel <as...@redhat.com>.
----- Original Message -----
> From: "Christopher Schultz" <ch...@christopherschultz.net>
> To: users@tomcat.apache.org
> Sent: Thursday, December 5, 2019 5:05:42 PM
> Subject: Re: Tomcat - No Fork for debugging?
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Alex,
>
> On 12/5/19 15:18, Alex Scheel wrote:
> > Hi all,
> >
> > I've tried searching to no avail.
> >
> > I'm working on a(nother) SSL adapter. However, I've had some issues
> > with it. There's a native component and I'm trying to tease apart
> > its relationship with why the client won't handshake. The stack
> > traces aren't overly helpful and I'd love to attach gdb to this and
> > set a few specific environment variables.
> >
> > Query:
> >
> > Does Tomcat have a mode where it won't fork to a different user and
> > will run with a limited number of threads? That'd greatly improve
> > my ability to debug. Something similar to `radiusd -X` or `sshd
> > -d`?
>
> You can accomplish this in two steps:
>
> 1. Change your <Connector> to have an <Executor> with only a single
> thread, or set maxThreads="1" on the <Connector> if you aren't
> separately configuring an <Executor>
>
> 2. From the command line, run:
>
> $ bin/catalina.sh run
I think that behaves better. Thanks!
> or
>
> X:> bin\catalina.bat run
>
> This will run Tomcat directly in the terminal window (or, on Windows,
> open a second terminal window where it will run).
>
> You can see stdout in that terminal, and you can press CTRL-C to kill
> the process.
>
> > Rationale:
> >
> > NSS has support for logging calls to its PKCS#11 interface to a
> > file, based on the presence of environment variables. When I set
> > these environment variables and directly call the JVM to start
> > Tomcat:
> >
> > # java -classpath $CLASSPATH $FLAGS
> > org.apache.catalina.startup.Bootstrap start
> >
> > I see it logging calls when the JDK starts up, but when I hit it
> > with wget on the TLS port, the resulting PKCS#11 calls aren't
> > logged. When launching in gdb, I get an error about /sbin/nologin
> > doesn't understand the -c option, which to me says that Bootstrap
> > is trying to fork and create a new shell (I'm running as root in a
> > VM and it wants to launch as the tomcat user), dropping my
> > environmental variables I want.
> >
> > Ideally (for debugging) I'd like to simplify this. Is there a more
> > direct entry route I can use perhaps?
>
> Is there a JSSE wrapper for NSS? You can just plug-in the crypto
> provider for the SSLContext instead of writing your own connector.
> Wait. You said "adapter". What kind of "adapter" are you writing?
Ah sorry, my terminology was loose there. I'm writing another
org.apache.tomcat.util.net.SSLImplementation implementation. Heh.
I'm the maintainer of JSS, a NSS wrapper in Java. This is mostly utilized
by the Dogtag project.
I also co-maintain TomcatJSS, a JSS adapter for Tomcat < 8.5. When
used with Tomcat >= 8.5, it uses JSSE's SSLEngine but initializes JSS
based on configuration in server.xml and use it for the keystore. Mostly
this is because we've not yet gotten around to adding a SSLEngine to JSS
(but will likely finish that work soon -- JSS was started at Netscape back
in the late 90s and predated the SSLEngine being added to JVM interfaces
in JDK version 5). RHEL 7 still ships with Tomcat 7, but RHEL 8 only has
Tomcat 9, so the hard requirement for a SSLEngine is new. We're getting
there :)
With FIPS certification requirements though, we can't use JSSE (since
it isn't FIPS certified). OpenJDK supports NSS via the Sun-PKCS11 provider
and has a SSLEngine implementation under the SunJSSE provider. It has
some restrictions on it that means it can't be used from Tomcat's JSSE
adapter just by swapping out the provider name (for one, wrapping the
KeyManager like Tomcat does isn't allowed).
I was trying to see if the JDK's SSLEngine would work for us. On RHEL,
we FIPS certify our shipped NSS release so the JDK's Sun-PKCS11-NSS-FIPS
provider is FIPS certified transitively (since it does no crypto... :).
Since I'm not a Tomcat developer (and am more familiar with TomcatJSS),
I stubbed out a new SSLImplementation that uses SunJSSE so I could
bypass the parts of the Tomcat JSSEImplementation that don't work with
the JDK SunJSSE provider.
However, currently on RHEL the SunJSSE provider is a bit broken, so this
FIPSImplementation isn't really useful. If you're interested, I'd like
to get Tomcat to a place where it supports FIPS mode JDK without the need
for a separate implementation, but first I need a working FIPS-mode JDK
(currently it doesn't handshake)... ...and we're probably going the
route of finishing JSS anyways.
HTH,
- Alex
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl3pfzUACgkQHPApP6U8
> pFjVLRAAulhpc4XyarJTFJQ3Iru2MM+X3LrFCmcvpuPLI/j/UXFtEKRh6Wqgx6s1
> yt9E++3LvQvBFaL/NO5a2LDncS186SjUliMorrF4Yc3f0Cc5iaCgzaZe4uyj4RIg
> MYidArw56r+FViY2DnHxBLzN4uqANYdnFVGonMei9puvJmBsoQk9LNSOl2RuRJiI
> +zg1tjUyjcoSPC9mR/hYk3CI+H2N9FoKKvpkYQKHzyUG+sz4yRSJkx9VfMKHzfAF
> xU6DZpKyNRI9Oes4W+B0FMBWF0+nH67O1QI6nnqSjeOuyJcr51kZBswk0kA1iDlJ
> 6iAAMp++rB7sSof5PMNId6yKTcda4djolUwFuhoyphORDhd2N8B1VfwVqdo1c2Z1
> vT/8smDjcLiCmW52avY066ujxr3F4BSkXmCCelncfCVpa/uj9hAx1Oaic97bPmJG
> KZSijm2kTPRMa6DhvHHg1GmcAvmzfwdpr6YBhxBUJpR+3ZhT+luQpRxvRJREFqnP
> pNVICNZ/hWzrtc3gHMZJEH7mcTHKgwU9gD/ctDLnZ7qN5r6LBBfsy17Y/oTnydPN
> bJ4/CAMZNVxJMyoew00ZixnCyzRB82QNs2cQClVssSbih5765YEJ/AfssCkcDAZh
> rPxcWcu+tWWJiih1NW4oGIxvAXc4jn8h8/c+C6R7iCnTuXuJmz4=
> =2O0L
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat - No Fork for debugging?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Alex,
On 12/5/19 15:18, Alex Scheel wrote:
> Hi all,
>
> I've tried searching to no avail.
>
> I'm working on a(nother) SSL adapter. However, I've had some issues
> with it. There's a native component and I'm trying to tease apart
> its relationship with why the client won't handshake. The stack
> traces aren't overly helpful and I'd love to attach gdb to this and
> set a few specific environment variables.
>
> Query:
>
> Does Tomcat have a mode where it won't fork to a different user and
> will run with a limited number of threads? That'd greatly improve
> my ability to debug. Something similar to `radiusd -X` or `sshd
> -d`?
You can accomplish this in two steps:
1. Change your <Connector> to have an <Executor> with only a single
thread, or set maxThreads="1" on the <Connector> if you aren't
separately configuring an <Executor>
2. From the command line, run:
$ bin/catalina.sh run
or
X:> bin\catalina.bat run
This will run Tomcat directly in the terminal window (or, on Windows,
open a second terminal window where it will run).
You can see stdout in that terminal, and you can press CTRL-C to kill
the process.
> Rationale:
>
> NSS has support for logging calls to its PKCS#11 interface to a
> file, based on the presence of environment variables. When I set
> these environment variables and directly call the JVM to start
> Tomcat:
>
> # java -classpath $CLASSPATH $FLAGS
> org.apache.catalina.startup.Bootstrap start
>
> I see it logging calls when the JDK starts up, but when I hit it
> with wget on the TLS port, the resulting PKCS#11 calls aren't
> logged. When launching in gdb, I get an error about /sbin/nologin
> doesn't understand the -c option, which to me says that Bootstrap
> is trying to fork and create a new shell (I'm running as root in a
> VM and it wants to launch as the tomcat user), dropping my
> environmental variables I want.
>
> Ideally (for debugging) I'd like to simplify this. Is there a more
> direct entry route I can use perhaps?
Is there a JSSE wrapper for NSS? You can just plug-in the crypto
provider for the SSLContext instead of writing your own connector.
Wait. You said "adapter". What kind of "adapter" are you writing?
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl3pfzUACgkQHPApP6U8
pFjVLRAAulhpc4XyarJTFJQ3Iru2MM+X3LrFCmcvpuPLI/j/UXFtEKRh6Wqgx6s1
yt9E++3LvQvBFaL/NO5a2LDncS186SjUliMorrF4Yc3f0Cc5iaCgzaZe4uyj4RIg
MYidArw56r+FViY2DnHxBLzN4uqANYdnFVGonMei9puvJmBsoQk9LNSOl2RuRJiI
+zg1tjUyjcoSPC9mR/hYk3CI+H2N9FoKKvpkYQKHzyUG+sz4yRSJkx9VfMKHzfAF
xU6DZpKyNRI9Oes4W+B0FMBWF0+nH67O1QI6nnqSjeOuyJcr51kZBswk0kA1iDlJ
6iAAMp++rB7sSof5PMNId6yKTcda4djolUwFuhoyphORDhd2N8B1VfwVqdo1c2Z1
vT/8smDjcLiCmW52avY066ujxr3F4BSkXmCCelncfCVpa/uj9hAx1Oaic97bPmJG
KZSijm2kTPRMa6DhvHHg1GmcAvmzfwdpr6YBhxBUJpR+3ZhT+luQpRxvRJREFqnP
pNVICNZ/hWzrtc3gHMZJEH7mcTHKgwU9gD/ctDLnZ7qN5r6LBBfsy17Y/oTnydPN
bJ4/CAMZNVxJMyoew00ZixnCyzRB82QNs2cQClVssSbih5765YEJ/AfssCkcDAZh
rPxcWcu+tWWJiih1NW4oGIxvAXc4jn8h8/c+C6R7iCnTuXuJmz4=
=2O0L
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org