You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2019/03/09 13:11:07 UTC

[Bug 63251] New: [JDK-8194653] Deadlock in FileSystems.getDefault on catalina startup

https://bz.apache.org/bugzilla/show_bug.cgi?id=63251

            Bug ID: 63251
           Summary: [JDK-8194653] Deadlock in FileSystems.getDefault on
                    catalina startup
           Product: Tomcat 9
           Version: 9.0.14
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P2
         Component: Catalina
          Assignee: dev@tomcat.apache.org
          Reporter: dmitry.bedrin@gmail.com
  Target Milestone: -----

Tomcat 9.0.14 fails to start with default logging settings intermittently.
Thread dump shows a dead lock (known issue in JRE:
https://bugs.openjdk.java.net/browse/JDK-8194653)

Other libraries like Gatling, Payara and others workaround it by calling
FileSystems.getDefault from main thread before starting other threads.

Full thread dump OpenJDK 64-Bit Server VM (25.201-b10 mixed mode):

"FileHandlerLogFilesCleaner-1" #11 daemon prio=5 os_prio=0
tid=0x00007fcfc41c4000 nid=0x27 waiting for monitor entry [0x00007fcf8fffa000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at java.lang.Runtime.loadLibrary0(Runtime.java:862)
        - waiting to lock <0x000000008001a690> (a java.lang.Runtime)
        at java.lang.System.loadLibrary(System.java:1122)
        at sun.nio.fs.UnixNativeDispatcher$1.run(UnixNativeDispatcher.java:573)
        at sun.nio.fs.UnixNativeDispatcher$1.run(UnixNativeDispatcher.java:571)
        at java.security.AccessController.doPrivileged(Native Method)
        at
sun.nio.fs.UnixNativeDispatcher.<clinit>(UnixNativeDispatcher.java:571)
        at sun.nio.fs.UnixFileSystem.<init>(UnixFileSystem.java:67)
        at sun.nio.fs.LinuxFileSystem.<init>(LinuxFileSystem.java:39)
        at
sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:46)
        at
sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:39)
        at
sun.nio.fs.UnixFileSystemProvider.<init>(UnixFileSystemProvider.java:56)
        at
sun.nio.fs.LinuxFileSystemProvider.<init>(LinuxFileSystemProvider.java:41)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.lang.Class.newInstance(Class.java:442)
        at
sun.nio.fs.DefaultFileSystemProvider.createProvider(DefaultFileSystemProvider.java:48)
        at
sun.nio.fs.DefaultFileSystemProvider.create(DefaultFileSystemProvider.java:63)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:108)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.access$000(FileSystems.java:89)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:98)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:96)
        at java.security.AccessController.doPrivileged(Native Method)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:96)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(FileSystems.java:90)
        at java.nio.file.FileSystems.getDefault(FileSystems.java:176)
        at java.io.File.toPath(File.java:2234)
        - locked <0x0000000080b1c268> (a java.io.File)
        at
org.apache.juli.FileHandler.streamFilesForDelete(FileHandler.java:559)
        at org.apache.juli.FileHandler.lambda$clean$0(FileHandler.java:546)
        at org.apache.juli.FileHandler$$Lambda$1/20132171.run(Unknown Source)
        at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"AsyncFileHandlerWriter-1259475182" #10 daemon prio=5 os_prio=0
tid=0x00007fcfc4141800 nid=0x25 waiting on condition [0x00007fcfb41f8000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x000000008011cac8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at
java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
        at
java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
        at
org.apache.juli.AsyncFileHandler$LoggerThread.run(AsyncFileHandler.java:159)

"Service Thread" #7 daemon prio=9 os_prio=0 tid=0x00007fcfc40cd800 nid=0x1c
runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C1 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007fcfc40c6800 nid=0x1b
waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007fcfc40c4000 nid=0x1a
waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007fcfc40c2000 nid=0x19
waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007fcfc407a800 nid=0x18 in
Object.wait() [0x00007fcfb47fe000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000080008ed0> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
        - locked <0x0000000080008ed0> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)

"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007fcfc4078000 nid=0x17
in Object.wait() [0x00007fcfc81c6000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000080006bf8> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:502)
        at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
        - locked <0x0000000080006bf8> (a java.lang.ref.Reference$Lock)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

"main" #1 prio=5 os_prio=0 tid=0x00007fcfc400a800 nid=0x15 in Object.wait()
[0x00007fcfcb86e000]
   java.lang.Thread.State: RUNNABLE
        at java.nio.file.FileSystems.getDefault(FileSystems.java:176)
        at java.nio.file.Paths.get(Paths.java:138)
        at sun.misc.Launcher$ExtClassLoader.findLibrary(Launcher.java:235)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1830)
        at java.lang.Runtime.loadLibrary0(Runtime.java:870)
        - locked <0x000000008001a690> (a java.lang.Runtime)
        at java.lang.System.loadLibrary(System.java:1122)
        at sun.security.ec.SunEC$1.run(SunEC.java:60)
        at sun.security.ec.SunEC$1.run(SunEC.java:58)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.security.ec.SunEC.<clinit>(SunEC.java:58)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.lang.Class.newInstance(Class.java:442)
        at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:221)
        at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:206)
        at java.security.AccessController.doPrivileged(Native Method)
        at
sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:206)
        at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:187)
        - locked <0x000000008005db60> (a sun.security.jca.ProviderConfig)
        at sun.security.jca.ProviderList.getProvider(ProviderList.java:233)
        at sun.security.jca.ProviderList.getIndex(ProviderList.java:263)
        at
sun.security.jca.ProviderList.getProviderConfig(ProviderList.java:247)
        at sun.security.jca.ProviderList.getProvider(ProviderList.java:253)
        at java.net.URL.getURLStreamHandler(URL.java:1159)
        at java.net.URL.<init>(URL.java:421)
        at sun.misc.URLClassPath$JarLoader.<init>(URLClassPath.java:822)
        at sun.misc.URLClassPath$3.run(URLClassPath.java:575)
        at sun.misc.URLClassPath$3.run(URLClassPath.java:565)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.misc.URLClassPath.getLoader(URLClassPath.java:564)
        at sun.misc.URLClassPath.getLoader(URLClassPath.java:529)
        - locked <0x00000000808c3b48> (a sun.misc.URLClassPath)
        at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:494)
        - locked <0x00000000808c3b48> (a sun.misc.URLClassPath)
        at sun.misc.URLClassPath.getResource(URLClassPath.java:248)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        - locked <0x00000000808cf780> (a java.lang.Object)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at org.apache.catalina.startup.Bootstrap.init(Bootstrap.java:263)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:461)
        - locked <0x000000008086db40> (a java.lang.Object)

"VM Thread" os_prio=0 tid=0x00007fcfc406e800 nid=0x16 runnable 

"VM Periodic Task Thread" os_prio=0 tid=0x00007fcfc40d0800 nid=0x1d waiting on
condition 

JNI global references: 322

Heap
 def new generation   total 157248K, used 13977K [0x0000000080000000,
0x000000008aaa0000, 0x00000000aaaa0000)
  eden space 139776K,  10% used [0x0000000080000000, 0x0000000080da67d8,
0x0000000088880000)
  from space 17472K,   0% used [0x0000000088880000, 0x0000000088880000,
0x0000000089990000)
  to   space 17472K,   0% used [0x0000000089990000, 0x0000000089990000,
0x000000008aaa0000)
 tenured generation   total 349568K, used 0K [0x00000000aaaa0000,
0x00000000c0000000, 0x0000000100000000)
   the space 349568K,   0% used [0x00000000aaaa0000, 0x00000000aaaa0000,
0x00000000aaaa0200, 0x00000000c0000000)
 Metaspace       used 6094K, capacity 6240K, committed 6400K, reserved 1056768K
  class space    used 664K, capacity 741K, committed 768K, reserved 1048576K

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 63251] [JDK-8194653] Deadlock in FileSystems.getDefault on catalina startup

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63251

Mark Thomas <ma...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #1 from Mark Thomas <ma...@apache.org> ---
Thanks for the report and the stack trace showing the problem.

Fixed in:
- master for 9.0.18 onwards
- 8.5.x for 8.5.40 onwards
- 7.0.x for 7.0.94 onwards

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 63251] [JDK-8194653] Deadlock in FileSystems.getDefault on catalina startup

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63251

--- Comment #3 from harristr@gmail.com ---
(In reply to harristr from comment #2)
> The work-around of pre-loading the default FileSystem does not appear to
> solve the problem for logging, or maybe it only reduced the frequency of the
> occurrence.  I am still seeing the following stacktrace from a Spring Boot
> Microservice running Tomcat 9.0.22 with JDK 1.8.0_222 64-bit on RedHat 7.5:

After further investigation I think this work-around actually CAUSED the
"deadlock" I am experiencing.  I never saw this using Tomcat 9.0.17, it only
manifested after an upgrade directly to 9.0.22.

In an environment such as mine (a SpringBoot Microservice), Tomcat is NOT
running in the main thread or starting first, Spring Boot is.  This means the
probability of reaching the "deadlock" is higher since Spring Boot is the one
running first and Tomcat is coming up in a different thread but is now trying
to prime the FileSystem right away.

I included more of the thread dump below.


"background-preinit" #9 prio=5 os_prio=0 tid=0x0000000002d11800 nid=0x726
waiting for monitor entry [0x00007f26ddd01000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at java.lang.Runtime.loadLibrary0(Runtime.java:862)
        - waiting to lock <0x00000000c010be00> (a java.lang.Runtime)
        at java.lang.System.loadLibrary(System.java:1122)
        at sun.nio.fs.UnixNativeDispatcher$1.run(UnixNativeDispatcher.java:573)
        at sun.nio.fs.UnixNativeDispatcher$1.run(UnixNativeDispatcher.java:571)
        at java.security.AccessController.doPrivileged(Native Method)
        at
sun.nio.fs.UnixNativeDispatcher.<clinit>(UnixNativeDispatcher.java:571)
        at sun.nio.fs.UnixFileSystem.<init>(UnixFileSystem.java:67)
        at sun.nio.fs.LinuxFileSystem.<init>(LinuxFileSystem.java:39)
        at
sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:46)
        at
sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:39)
        at
sun.nio.fs.UnixFileSystemProvider.<init>(UnixFileSystemProvider.java:56)
        at
sun.nio.fs.LinuxFileSystemProvider.<init>(LinuxFileSystemProvider.java:41)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.lang.Class.newInstance(Class.java:442)
        at
sun.nio.fs.DefaultFileSystemProvider.createProvider(DefaultFileSystemProvider.java:48)
        at
sun.nio.fs.DefaultFileSystemProvider.create(DefaultFileSystemProvider.java:63)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:108)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.access$000(FileSystems.java:89)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:98)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:96)
        at java.security.AccessController.doPrivileged(Native Method)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:96)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(FileSystems.java:90)
        at java.nio.file.FileSystems.getDefault(FileSystems.java:176)
        at org.apache.juli.logging.LogFactory.<init>(LogFactory.java:85)
        at org.apache.juli.logging.LogFactory.<clinit>(LogFactory.java:66)
        at
org.apache.catalina.mbeans.MBeanFactory.<clinit>(MBeanFactory.java:59)
        at
org.springframework.boot.autoconfigure.BackgroundPreinitializer$MBeanFactoryInitializer.run(BackgroundPreinitializer.java:139)
        at
org.springframework.boot.autoconfigure.BackgroundPreinitializer$1.runSafely(BackgroundPreinitializer.java:102)
        at
org.springframework.boot.autoconfigure.BackgroundPreinitializer$1.run(BackgroundPreinitializer.java:94)
        at java.lang.Thread.run(Thread.java:748)

"Service Thread" #8 daemon prio=9 os_prio=0 tid=0x0000000000fa0000 nid=0x6d7
runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C1 CompilerThread1" #7 daemon prio=9 os_prio=0 tid=0x0000000000f90000
nid=0x6d6 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 tid=0x0000000000f98800
nid=0x6d4 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x0000000000ebf800 nid=0x6d0
waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x0000000000e95000 nid=0x6c9 in
Object.wait() [0x00007f26e4c2d000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000c0166a10> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
        - locked <0x00000000c0166a10> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)

"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x0000000000e8a800
nid=0x6c8 in Object.wait() [0x00007f26e4c6e000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000c0299c90> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:502)
        at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
        - locked <0x00000000c0299c90> (a java.lang.ref.Reference$Lock)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

"main" #1 prio=5 os_prio=0 tid=0x0000000000b52000 nid=0x687 in Object.wait()
[0x00007f26fee02000]
   java.lang.Thread.State: RUNNABLE
        at java.nio.file.FileSystems.getDefault(FileSystems.java:176)
        at java.nio.file.Paths.get(Paths.java:138)
        at sun.misc.Launcher$ExtClassLoader.findLibrary(Launcher.java:235)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1830)
        at java.lang.Runtime.loadLibrary0(Runtime.java:870)
        - locked <0x00000000c010be00> (a java.lang.Runtime)
        at java.lang.System.loadLibrary(System.java:1122)
        at sun.security.ec.SunEC$1.run(SunEC.java:60)
        at sun.security.ec.SunEC$1.run(SunEC.java:58)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.security.ec.SunEC.<clinit>(SunEC.java:58)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.lang.Class.newInstance(Class.java:442)
        at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:221)
        at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:206)
        at java.security.AccessController.doPrivileged(Native Method)
        at
sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:206)
        at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:187)
        - locked <0x00000000c055a6a0> (a sun.security.jca.ProviderConfig)
        at sun.security.jca.ProviderList.getProvider(ProviderList.java:233)
        at sun.security.jca.ProviderList.getService(ProviderList.java:331)
        at sun.security.jca.GetInstance.getInstance(GetInstance.java:157)
        at javax.net.ssl.SSLContext.getInstance(SSLContext.java:156)
        at javax.net.ssl.SSLContext.getDefault(SSLContext.java:96)
        - locked <0x00000000eb60aa50> (a java.lang.Class for
javax.net.ssl.SSLContext)
        at javax.net.ssl.SSLSocketFactory.getDefault(SSLSocketFactory.java:122)
        - locked <0x00000000eb608ec8> (a java.lang.Class for
javax.net.ssl.SSLSocketFactory)
        at
javax.net.ssl.HttpsURLConnection.getDefaultSSLSocketFactory(HttpsURLConnection.java:332)
        at javax.net.ssl.HttpsURLConnection.<init>(HttpsURLConnection.java:289)
        at
sun.net.www.protocol.https.HttpsURLConnectionImpl.<init>(HttpsURLConnectionImpl.java:99)
        at sun.net.www.protocol.https.Handler.openConnection(Handler.java:62)
        at sun.net.www.protocol.https.Handler.openConnection(Handler.java:57)
        at java.net.URL.openConnection(URL.java:991)
        at
org.springframework.http.client.SimpleClientHttpRequestFactory.openConnection(SimpleClientHttpRequestFactory.java:187)
        at
org.springframework.http.client.SimpleClientHttpRequestFactory.createRequest(SimpleClientHttpRequestFactory.java:145)
        at
org.springframework.http.client.support.HttpAccessor.createRequest(HttpAccessor.java:87)
        at
org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:731)
        at
org.springframework.web.client.RestTemplate.execute(RestTemplate.java:691)
        at
org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:618)
        at
com.sas.commons.consul.util.ExchangeUtil.internalExchange(ExchangeUtil.java:161)
        at
com.sas.commons.consul.util.ExchangeUtil.callExchangeMethod(ExchangeUtil.java:118)
        at
com.sas.commons.consul.util.ExchangeUtil.getForObjectWithRetry(ExchangeUtil.java:320)
        at
com.sas.commons.consul.impl.ServiceRegistrationImpl.getServiceDetails(ServiceRegistrationImpl.java:160)
        at
com.sas.commons.consul.service.ConsulServiceResolver.getNodes(ConsulServiceResolver.java:249)
        at
com.sas.commons.consul.service.ConsulServiceResolver.resolveServiceWithTags(ConsulServiceResolver.java:205)
        at
com.sas.commons.consul.service.ConsulServiceResolver.resolveService(ConsulServiceResolver.java:180)
        at
com.sas.commons.consul.service.ConsulServiceResolver.resolveService(ConsulServiceResolver.java:165)
        at
com.sas.commons.rest.boot.initialization.env.VaultBootstrapPropertyLoader.resolveVaultFromConsul(VaultBootstrapPropertyLoader.java:65)
        at
com.sas.commons.rest.boot.initialization.env.VaultBootstrapPropertyLoader.load(VaultBootstrapPropertyLoader.java:44)
        at
com.sas.commons.rest.boot.initialization.env.BootstrapEnvironmentPropertyInitializer.onApplicationEvent(BootstrapEnvironmentPropertyInitializer.java:63)
        at
com.sas.commons.rest.boot.initialization.env.BootstrapEnvironmentPropertyInitializer.onApplicationEvent(BootstrapEnvironmentPropertyInitializer.java:30)
        at
org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
        at
org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
        at
org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
        at
org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:127)
        at
org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:76)
        at
org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:53)
        at
org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:341)
        at
org.springframework.boot.SpringApplication.run(SpringApplication.java:305)
        at
org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:140)
        - locked <0x00000000c05f83a0> (a
java.util.concurrent.atomic.AtomicBoolean)
        at
org.springframework.cloud.bootstrap.BootstrapApplicationListener.bootstrapServiceContext(BootstrapApplicationListener.java:203)
        at
org.springframework.cloud.bootstrap.BootstrapApplicationListener.onApplicationEvent(BootstrapApplicationListener.java:114)
        at
org.springframework.cloud.bootstrap.BootstrapApplicationListener.onApplicationEvent(BootstrapApplicationListener.java:71)
        at
org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
        at
org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
        at
org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
        at
org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:127)
        at
org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:76)
        at
org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:53)
        at
org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:341)
        at
org.springframework.boot.SpringApplication.run(SpringApplication.java:305)
        at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1214)
        at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1203)
        at com.sas.transfer.Application.main(Application.java:19)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at
org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
        at
org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)

"VM Thread" os_prio=0 tid=0x0000000000e81000 nid=0x6c7 runnable

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 63251] [JDK-8194653] Deadlock in FileSystems.getDefault on catalina startup

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63251

harristr@gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|9.0.14                      |9.0.22

--- Comment #2 from harristr@gmail.com ---
The work-around of pre-loading the default FileSystem does not appear to solve
the problem for logging, or maybe it only reduced the frequency of the
occurrence.  I am still seeing the following stacktrace from a Spring Boot
Microservice running Tomcat 9.0.22 with JDK 1.8.0_222 64-bit on RedHat 7.5:

2019-09-07 15:39:26
Full thread dump OpenJDK 64-Bit Server VM (25.222-b10 mixed mode):

"background-preinit" #9 prio=5 os_prio=0 tid=0x0000000002d11800 nid=0x726
waiting for monitor entry [0x00007f26ddd01000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at java.lang.Runtime.loadLibrary0(Runtime.java:862)
        - waiting to lock <0x00000000c010be00> (a java.lang.Runtime)
        at java.lang.System.loadLibrary(System.java:1122)
        at sun.nio.fs.UnixNativeDispatcher$1.run(UnixNativeDispatcher.java:573)
        at sun.nio.fs.UnixNativeDispatcher$1.run(UnixNativeDispatcher.java:571)
        at java.security.AccessController.doPrivileged(Native Method)
        at
sun.nio.fs.UnixNativeDispatcher.<clinit>(UnixNativeDispatcher.java:571)
        at sun.nio.fs.UnixFileSystem.<init>(UnixFileSystem.java:67)
        at sun.nio.fs.LinuxFileSystem.<init>(LinuxFileSystem.java:39)
        at
sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:46)
        at
sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:39)
        at
sun.nio.fs.UnixFileSystemProvider.<init>(UnixFileSystemProvider.java:56)
        at
sun.nio.fs.LinuxFileSystemProvider.<init>(LinuxFileSystemProvider.java:41)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.lang.Class.newInstance(Class.java:442)
        at
sun.nio.fs.DefaultFileSystemProvider.createProvider(DefaultFileSystemProvider.java:48)
        at
sun.nio.fs.DefaultFileSystemProvider.create(DefaultFileSystemProvider.java:63)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:108)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.access$000(FileSystems.java:89)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:98)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:96)
        at java.security.AccessController.doPrivileged(Native Method)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:96)
        at
java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(FileSystems.java:90)
        at java.nio.file.FileSystems.getDefault(FileSystems.java:176)
        at org.apache.juli.logging.LogFactory.<init>(LogFactory.java:85)
        at org.apache.juli.logging.LogFactory.<clinit>(LogFactory.java:66)
        at
org.apache.catalina.mbeans.MBeanFactory.<clinit>(MBeanFactory.java:59)
        at
org.springframework.boot.autoconfigure.BackgroundPreinitializer$MBeanFactoryInitializer.run(BackgroundPreinitializer.java:139)
        at
org.springframework.boot.autoconfigure.BackgroundPreinitializer$1.runSafely(BackgroundPreinitializer.java:102)
        at
org.springframework.boot.autoconfigure.BackgroundPreinitializer$1.run(BackgroundPreinitializer.java:94)
        at java.lang.Thread.run(Thread.java:748)

"main" #1 prio=5 os_prio=0 tid=0x0000000000b52000 nid=0x687 in Object.wait()
[0x00007f26fee02000]
   java.lang.Thread.State: RUNNABLE
        at java.nio.file.FileSystems.getDefault(FileSystems.java:176)
        at java.nio.file.Paths.get(Paths.java:138)
        at sun.misc.Launcher$ExtClassLoader.findLibrary(Launcher.java:235)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1830)
        at java.lang.Runtime.loadLibrary0(Runtime.java:870)
        - locked <0x00000000c010be00> (a java.lang.Runtime)
        at java.lang.System.loadLibrary(System.java:1122)
        at sun.security.ec.SunEC$1.run(SunEC.java:60)
        at sun.security.ec.SunEC$1.run(SunEC.java:58)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.security.ec.SunEC.<clinit>(SunEC.java:58)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.lang.Class.newInstance(Class.java:442)
        at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:221)
        at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:206)
        at java.security.AccessController.doPrivileged(Native Method)
        at
sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:206)
        at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:187)
        - locked <0x00000000c055a6a0> (a sun.security.jca.ProviderConfig)
        at sun.security.jca.ProviderList.getProvider(ProviderList.java:233)
        at sun.security.jca.ProviderList.getService(ProviderList.java:331)
        at sun.security.jca.GetInstance.getInstance(GetInstance.java:157)
        at javax.net.ssl.SSLContext.getInstance(SSLContext.java:156)
        at javax.net.ssl.SSLContext.getDefault(SSLContext.java:96)
        - locked <0x00000000eb60aa50> (a java.lang.Class for
javax.net.ssl.SSLContext)
        at javax.net.ssl.SSLSocketFactory.getDefault(SSLSocketFactory.java:122)
        - locked <0x00000000eb608ec8> (a java.lang.Class for
javax.net.ssl.SSLSocketFactory)
        at
javax.net.ssl.HttpsURLConnection.getDefaultSSLSocketFactory(HttpsURLConnection.java:332)
        at javax.net.ssl.HttpsURLConnection.<init>(HttpsURLConnection.java:289)
        at
sun.net.www.protocol.https.HttpsURLConnectionImpl.<init>(HttpsURLConnectionImpl.java:99)
        at sun.net.www.protocol.https.Handler.openConnection(Handler.java:62)
        at sun.net.www.protocol.https.Handler.openConnection(Handler.java:57)
        at java.net.URL.openConnection(URL.java:991)
        at
org.springframework.http.client.SimpleClientHttpRequestFactory.openConnection(SimpleClientHttpRequestFactory.java:187)
        at
org.springframework.http.client.SimpleClientHttpRequestFactory.createRequest(SimpleClientHttpRequestFactory.java:145)
        at
org.springframework.http.client.support.HttpAccessor.createRequest(HttpAccessor.java:87)
        at
org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:731)
        at
org.springframework.web.client.RestTemplate.execute(RestTemplate.java:691)
        at
org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:618)

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 63251] [JDK-8194653] Deadlock in FileSystems.getDefault on catalina startup

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63251

harristr@gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |harristr@gmail.com

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 63251] [JDK-8194653] Deadlock in FileSystems.getDefault on catalina startup

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63251

--- Comment #4 from Mark Thomas <ma...@apache.org> ---
It would be worth testing with Java 8 u231 when released - it is reported as
fixed in that version.

Meanwhile, you can see if the Spring Boot team is willing to add the workaround
to Boot.

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org