You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Ignite TC Bot (Jira)" <ji...@apache.org> on 2021/01/12 13:14:00 UTC
[jira] [Commented] (IGNITE-13960) Starvation in mgmt pool caused by
MetadataTask execution
[ https://issues.apache.org/jira/browse/IGNITE-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263322#comment-17263322 ]
Ignite TC Bot commented on IGNITE-13960:
----------------------------------------
{panel:title=Branch: [pull/8647/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/8647/head] Base: [master] : New Tests (1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#00008b}Cache 1{color} [[tests 1|https://ci.ignite.apache.org/viewLog.html?buildId=5824561]]
* {color:#013220}IgniteRestHandlerTestSuite: GridCacheMetadataCommandTest.cacheMetadataWithMultupleThreads - PASSED{color}
{panel}
[TeamCity *--> Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=5824605&buildTypeId=IgniteTests24Java8_RunAll]
> Starvation in mgmt pool caused by MetadataTask execution
> ---------------------------------------------------------
>
> Key: IGNITE-13960
> URL: https://issues.apache.org/jira/browse/IGNITE-13960
> Project: Ignite
> Issue Type: Bug
> Components: compute
> Affects Versions: 2.9.1
> Reporter: Pavel Vinokurov
> Assignee: Pavel Vinokurov
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> *Issue:*
> Requesting cache metadata from multiple threads causes starvation in the mgmt pool.
> *Root Cause:*
> From the mgmt pool GridCacheCommandHandler.MetadataJob calls GridCacheQueryManager#sqlMetadata() and GridClosureProcessor#callAsyncNoFailover().get() that executes and waits an another internal task. The job response of this task should be also handled from the mgmt pool. It causes starvation.
> *Proposed Fix:*
> Make GridCacheQueryManager#sqlMetadata() asynchronous and apply continuation for GridCacheCommandHandler.MetadataJob to release a mgmt thread for the time of completing the future returned by sqlMetadata().
> Attached threads with hanging threads:
> {code:java}
> "mgmt-#10633" #14311 prio=5 os_prio=0 tid=0x0000560c79117000 nid=0x134c6 waiting on condition [0x00007f15baa77000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.sqlMetadata(GridCacheQueryManager.java:1803)
> at org.apache.ignite.internal.processors.rest.handlers.cache.GridCacheCommandHandler$MetadataJob.execute(GridCacheCommandHandler.java:1123)
> at org.apache.ignite.internal.processors.rest.handlers.cache.GridCacheCommandHandler$MetadataJob.execute(GridCacheCommandHandler.java:1088)
> at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:567)
> at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7069)
> at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:561)
> at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:490)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
> at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1270)
> at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:2088)
> at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1635)
> at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1255)
> at org.apache.ignite.internal.managers.communication.GridIoManager.access$4300(GridIoManager.java:144)
> at org.apache.ignite.internal.managers.communication.GridIoManager$8.execute(GridIoManager.java:1144)
> at org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:50)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> "mgmt-#81" #270 prio=5 os_prio=0 tid=0x0000562323c3c800 nid=0x592 waiting on condition [0x00007fba5f378000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor$ClientChangeGlobalStateComputeRequest.run(GridClusterStateProcessor.java:1979)
> at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4.execute(GridClosureProcessor.java:1943)
> at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:567)
> at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7069)
> at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:561)
> at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:490)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
> at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1270)
> at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:2088)
> at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1635)
> at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1255)
> at org.apache.ignite.internal.managers.communication.GridIoManager.access$4300(GridIoManager.java:144)
> at org.apache.ignite.internal.managers.communication.GridIoManager$8.execute(GridIoManager.java:1144)
> at org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:50)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)