You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Xintong Song (Jira)" <ji...@apache.org> on 2020/12/12 11:56:00 UTC
[jira] [Updated] (FLINK-20266) New Sources prevent JVM shutdown
when running a job
[ https://issues.apache.org/jira/browse/FLINK-20266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xintong Song updated FLINK-20266:
---------------------------------
Fix Version/s: 1.11.3
> New Sources prevent JVM shutdown when running a job
> ---------------------------------------------------
>
> Key: FLINK-20266
> URL: https://issues.apache.org/jira/browse/FLINK-20266
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
> Affects Versions: 1.12.0
> Reporter: Till Rohrmann
> Assignee: Stephan Ewen
> Priority: Blocker
> Fix For: 1.12.0, 1.11.3
>
>
> When trying out the new {{FileSource}} I noticed that the jobs which I started from my IDE won't properly terminate. To be more precise the spawned JVM for the jobs wouldn't properly terminate. I cannot really tell what the {{FileSource}} does differently, but when not using this source, the JVM terminates properly.
> The stack trace of the hanging JVM is
> {code}
> 2020-11-20 18:20:02
> Full thread dump OpenJDK 64-Bit Server VM (11.0.2+9 mixed mode):
> Threads class SMR info:
> _java_thread_list=0x00007fb5bc15f1b0, length=19, elements={
> 0x00007fb60d807000, 0x00007fb60d80c000, 0x00007fb60d80f000, 0x00007fb60d809000,
> 0x00007fb60d81a000, 0x00007fb61f00b000, 0x00007fb63d80e000, 0x00007fb61d826800,
> 0x00007fb61d829800, 0x00007fb61e800000, 0x00007fb63d95d800, 0x00007fb63e2f8800,
> 0x00007fb5ba37a800, 0x00007fb5afe1a800, 0x00007fb61dff6800, 0x00007fb63da49800,
> 0x00007fb63e8d0800, 0x00007fb5be001000, 0x00007fb5bb8a4000
> }
> "Reference Handler" #2 daemon prio=10 os_prio=31 cpu=10.05ms elapsed=86.35s tid=0x00007fb60d807000 nid=0x4b03 waiting on condition [0x00007000036e9000]
> java.lang.Thread.State: RUNNABLE
> at java.lang.ref.Reference.waitForReferencePendingList(java.base@11.0.2/Native Method)
> at java.lang.ref.Reference.processPendingReferences(java.base@11.0.2/Reference.java:241)
> at java.lang.ref.Reference$ReferenceHandler.run(java.base@11.0.2/Reference.java:213)
> "Finalizer" #3 daemon prio=8 os_prio=31 cpu=0.90ms elapsed=86.35s tid=0x00007fb60d80c000 nid=0x3803 in Object.wait() [0x00007000037ec000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(java.base@11.0.2/Native Method)
> - waiting on <0x0000000600204780> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(java.base@11.0.2/ReferenceQueue.java:155)
> - waiting to re-lock in wait() <0x0000000600204780> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(java.base@11.0.2/ReferenceQueue.java:176)
> at java.lang.ref.Finalizer$FinalizerThread.run(java.base@11.0.2/Finalizer.java:170)
> "Signal Dispatcher" #4 daemon prio=9 os_prio=31 cpu=0.31ms elapsed=86.34s tid=0x00007fb60d80f000 nid=0x4203 runnable [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread0" #5 daemon prio=9 os_prio=31 cpu=2479.36ms elapsed=86.34s tid=0x00007fb60d809000 nid=0x3f03 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> No compile task
> "C1 CompilerThread0" #8 daemon prio=9 os_prio=31 cpu=1412.88ms elapsed=86.34s tid=0x00007fb60d81a000 nid=0x3d03 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> No compile task
> "Sweeper thread" #9 daemon prio=9 os_prio=31 cpu=42.82ms elapsed=86.34s tid=0x00007fb61f00b000 nid=0xa803 runnable [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "Common-Cleaner" #10 daemon prio=8 os_prio=31 cpu=3.25ms elapsed=86.29s tid=0x00007fb63d80e000 nid=0x5703 in Object.wait() [0x0000700003cfb000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(java.base@11.0.2/Native Method)
> - waiting on <0x0000000600205aa0> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(java.base@11.0.2/ReferenceQueue.java:155)
> - waiting to re-lock in wait() <0x0000000600205aa0> (a java.lang.ref.ReferenceQueue$Lock)
> at jdk.internal.ref.CleanerImpl.run(java.base@11.0.2/CleanerImpl.java:148)
> at java.lang.Thread.run(java.base@11.0.2/Thread.java:834)
> at jdk.internal.misc.InnocuousThread.run(java.base@11.0.2/InnocuousThread.java:134)
> "JDWP Transport Listener: dt_socket" #11 daemon prio=10 os_prio=31 cpu=43.46ms elapsed=86.27s tid=0x00007fb61d826800 nid=0xa603 runnable [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "JDWP Event Helper Thread" #12 daemon prio=10 os_prio=31 cpu=220.06ms elapsed=86.27s tid=0x00007fb61d829800 nid=0x5e03 runnable [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "JDWP Command Reader" #13 daemon prio=10 os_prio=31 cpu=27.26ms elapsed=86.27s tid=0x00007fb61e800000 nid=0x6103 runnable [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "Service Thread" #14 daemon prio=9 os_prio=31 cpu=0.06ms elapsed=86.19s tid=0x00007fb63d95d800 nid=0xa203 runnable [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "ForkJoinPool.commonPool-worker-19" #25 daemon prio=1 os_prio=31 cpu=2.00ms elapsed=84.76s tid=0x00007fb63e2f8800 nid=0x8003 waiting on condition [0x000070000584c000]
> java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11.0.2/Native Method)
> - parking to wait for <0x0000000600a81188> (a java.util.concurrent.ForkJoinPool)
> at java.util.concurrent.locks.LockSupport.park(java.base@11.0.2/LockSupport.java:194)
> at java.util.concurrent.ForkJoinPool.runWorker(java.base@11.0.2/ForkJoinPool.java:1628)
> at java.util.concurrent.ForkJoinWorkerThread.run(java.base@11.0.2/ForkJoinWorkerThread.java:177)
> "ForkJoinPool.commonPool-worker-23" #58 daemon prio=5 os_prio=31 cpu=0.19ms elapsed=84.12s tid=0x00007fb5ba37a800 nid=0xcc03 waiting on condition [0x0000700007197000]
> java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11.0.2/Native Method)
> - parking to wait for <0x0000000600a81188> (a java.util.concurrent.ForkJoinPool)
> at java.util.concurrent.locks.LockSupport.park(java.base@11.0.2/LockSupport.java:194)
> at java.util.concurrent.ForkJoinPool.runWorker(java.base@11.0.2/ForkJoinPool.java:1628)
> at java.util.concurrent.ForkJoinWorkerThread.run(java.base@11.0.2/ForkJoinWorkerThread.java:177)
> "ForkJoinPool.commonPool-worker-9" #59 daemon prio=5 os_prio=31 cpu=0.17ms elapsed=84.07s tid=0x00007fb5afe1a800 nid=0xcf03 waiting on condition [0x000070000729a000]
> java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11.0.2/Native Method)
> - parking to wait for <0x0000000600a81188> (a java.util.concurrent.ForkJoinPool)
> at java.util.concurrent.locks.LockSupport.park(java.base@11.0.2/LockSupport.java:194)
> at java.util.concurrent.ForkJoinPool.runWorker(java.base@11.0.2/ForkJoinPool.java:1628)
> at java.util.concurrent.ForkJoinWorkerThread.run(java.base@11.0.2/ForkJoinWorkerThread.java:177)
> "ForkJoinPool.commonPool-worker-27" #63 daemon prio=5 os_prio=31 cpu=0.72ms elapsed=83.97s tid=0x00007fb61dff6800 nid=0xd503 waiting on condition [0x00007000076a6000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11.0.2/Native Method)
> - parking to wait for <0x0000000600a81188> (a java.util.concurrent.ForkJoinPool)
> at java.util.concurrent.locks.LockSupport.parkUntil(java.base@11.0.2/LockSupport.java:275)
> at java.util.concurrent.ForkJoinPool.runWorker(java.base@11.0.2/ForkJoinPool.java:1619)
> at java.util.concurrent.ForkJoinWorkerThread.run(java.base@11.0.2/ForkJoinWorkerThread.java:177)
> "Cleaner-0" #185 daemon prio=8 os_prio=31 cpu=10.55ms elapsed=83.39s tid=0x00007fb63da49800 nid=0x1570b in Object.wait() [0x00007000093fd000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(java.base@11.0.2/Native Method)
> - waiting on <0x0000000600c5bd78> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(java.base@11.0.2/ReferenceQueue.java:155)
> - waiting to re-lock in wait() <0x0000000600c5bd78> (a java.lang.ref.ReferenceQueue$Lock)
> at jdk.internal.ref.CleanerImpl.run(java.base@11.0.2/CleanerImpl.java:148)
> at java.lang.Thread.run(java.base@11.0.2/Thread.java:834)
> at jdk.internal.misc.InnocuousThread.run(java.base@11.0.2/InnocuousThread.java:134)
> "ComponentClosingUtil" #232 prio=1 os_prio=31 cpu=0.39ms elapsed=81.78s tid=0x00007fb63e8d0800 nid=0x13107 waiting on condition [0x0000700004f31000]
> java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11.0.2/Native Method)
> - parking to wait for <0x00000006008fb930> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(java.base@11.0.2/LockSupport.java:194)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.2/AbstractQueuedSynchronizer.java:2081)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.2/ScheduledThreadPoolExecutor.java:1170)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.2/ScheduledThreadPoolExecutor.java:899)
> at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.2/ThreadPoolExecutor.java:1054)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.2/ThreadPoolExecutor.java:1114)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.2/ThreadPoolExecutor.java:628)
> at java.lang.Thread.run(java.base@11.0.2/Thread.java:834)
> "DestroyJavaVM" #282 prio=5 os_prio=31 cpu=2125.94ms elapsed=81.19s tid=0x00007fb5be001000 nid=0x1d03 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "Attach Listener" #291 daemon prio=9 os_prio=31 cpu=0.57ms elapsed=0.09s tid=0x00007fb5bb8a4000 nid=0x14207 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "VM Thread" os_prio=31 cpu=74.59ms elapsed=86.36s tid=0x00007fb63f807000 nid=0x3603 runnable
> "GC Thread#0" os_prio=31 cpu=51.88ms elapsed=86.38s tid=0x00007fb62d803800 nid=0x2e03 runnable
> "GC Thread#1" os_prio=31 cpu=50.32ms elapsed=85.97s tid=0x00007fb61d836800 nid=0x9e03 runnable
> "GC Thread#2" os_prio=31 cpu=45.79ms elapsed=85.97s tid=0x00007fb61d8b8000 nid=0x9d03 runnable
> "GC Thread#3" os_prio=31 cpu=46.08ms elapsed=85.97s tid=0x00007fb61d83a800 nid=0x9c03 runnable
> "GC Thread#4" os_prio=31 cpu=48.88ms elapsed=85.97s tid=0x00007fb61f01c000 nid=0x9a03 runnable
> "GC Thread#5" os_prio=31 cpu=44.32ms elapsed=85.97s tid=0x00007fb63d8e5000 nid=0x9803 runnable
> "GC Thread#6" os_prio=31 cpu=49.81ms elapsed=85.97s tid=0x00007fb63d8e6000 nid=0x9703 runnable
> "GC Thread#7" os_prio=31 cpu=51.55ms elapsed=85.97s tid=0x00007fb61d83b800 nid=0x6503 runnable
> "GC Thread#8" os_prio=31 cpu=46.05ms elapsed=85.97s tid=0x00007fb62d928000 nid=0x9403 runnable
> "GC Thread#9" os_prio=31 cpu=50.43ms elapsed=85.97s tid=0x00007fb62d929000 nid=0x9203 runnable
> "G1 Main Marker" os_prio=31 cpu=0.65ms elapsed=86.37s tid=0x00007fb62d83b000 nid=0x2f03 runnable
> "G1 Conc#0" os_prio=31 cpu=11.46ms elapsed=86.37s tid=0x00007fb62d83b800 nid=0x3203 runnable
> "G1 Conc#1" os_prio=31 cpu=11.43ms elapsed=85.39s tid=0x00007fb63da4e800 nid=0x6703 runnable
> "G1 Conc#2" os_prio=31 cpu=10.03ms elapsed=85.39s tid=0x00007fb63da4f800 nid=0x6803 runnable
> "G1 Refine#0" os_prio=31 cpu=8.22ms elapsed=86.37s tid=0x00007fb63e102000 nid=0x4e03 runnable
> "G1 Refine#1" os_prio=31 cpu=0.06ms elapsed=81.10s tid=0x00007fb5aeff4000 nid=0x1d50f runnable
> "G1 Young RemSet Sampling" os_prio=31 cpu=12.94ms elapsed=86.37s tid=0x00007fb63e102800 nid=0x3403 runnable
> "VM Periodic Task Thread" os_prio=31 cpu=57.93ms elapsed=86.19s tid=0x00007fb61e833800 nid=0xa003 waiting on condition
> JNI global refs: 55, weak refs: 15453
> {code}
> My environment is and I used the openjdk version "1.8.0_262" to run the job:
> IntelliJ IDEA 2020.2.3 (Community Edition)
> Build #IC-202.7660.26, built on October 6, 2020
> Runtime version: 11.0.8+10-b944.34 x86_64
> VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
> macOS 10.14.6
> GC: ParNew, ConcurrentMarkSweep
> Memory: 1979M
> Cores: 12
> Non-Bundled Plugins: CheckStyle-IDEA, com.jetbrains.performancePlugin, org.jetbrains.kotlin, org.intellij.scala
> cc [~sewen]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)