You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Chris Westin (JIRA)" <ji...@apache.org> on 2015/05/22 19:29:17 UTC
[jira] [Updated] (DRILL-3095) Memory Leak : Failure while closing
accountor.
[ https://issues.apache.org/jira/browse/DRILL-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Westin updated DRILL-3095:
--------------------------------
Fix Version/s: 1.1.0
> Memory Leak : Failure while closing accountor.
> ----------------------------------------------
>
> Key: DRILL-3095
> URL: https://issues.apache.org/jira/browse/DRILL-3095
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Affects Versions: 1.0.0
> Environment: f7f6efc525cd833ce1530deae32eb9ccb20b664a
> Reporter: Khurram Faraaz
> Assignee: Chris Westin
> Fix For: 1.1.0
>
>
> I am seeing a memory leak when i Cancel a long running query on sqlline. I am re running the query with assertion enabled, will add details after the second run is complete with assertions.
> Long running query was,
> {code}
> select key1, key2 from `twoKeyJsn.json`;
> {code}
> I did Ctrl-C when the above query was running on sqlline, and then issued the below query that returned correct results. And then I see there is a memory leak message in drillbit.log
> {code}
> 0: jdbc:drill:schema=dfs.tmp> select count(*) from `twoKeyJsn.json`;
> +------------+
> | EXPR$0 |
> +------------+
> | 26212355 |
> +------------+
> 1 row selected (14.734 seconds)
> {code}
> Stack trace from drillbit.log
> {code}
> 2015-05-15 00:59:01,951 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] WARN o.a.drill.exec.ops.SendingAccountor - Interrupted while waiting for send complete. Continuing to wait.
> java.lang.InterruptedException: null
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1301) ~[na:1.7.0_45]
> at java.util.concurrent.Semaphore.acquire(Semaphore.java:472) ~[na:1.7.0_45]
> at org.apache.drill.exec.ops.SendingAccountor.waitForSendComplete(SendingAccountor.java:48) ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.ops.FragmentContext.waitForSendComplete(FragmentContext.java:436) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.physical.impl.BaseRootExec.close(BaseRootExec.java:112) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.close(ScreenCreator.java:141) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:333) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:278) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> 2015-05-15 00:59:01,952 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 2aaabb50-3afd-2906-3f48-eb86a315a1f5:0:0: State change requested from CANCELLATION_REQUESTED --> FAILED for
> 2015-05-15 00:59:01,952 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 2aaabb50-3afd-2906-3f48-eb86a315a1f5:0:0: State change requested from FAILED --> FAILED for
> 2015-05-15 00:59:01,952 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 2aaabb50-3afd-2906-3f48-eb86a315a1f5:0:0: State change requested from FAILED --> FINISHED for
> 2015-05-15 00:59:01,956 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] ERROR o.a.d.c.exceptions.UserException - SYSTEM ERROR: java.lang.IllegalStateException: Failure while closing accountor. Expected private and shared pools to be set to initial values. However, one or more were not. Stats are
> zone init allocated delta
> private 1000000 918080 81920
> shared 9999000000 9999000000 0.
> Fragment 0:0
> [Error Id: 90ced8b1-b6db-438f-b193-b7634de31b81 on centos-03.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: java.lang.IllegalStateException: Failure while closing accountor. Expected private and shared pools to be set to initial values. However, one or more were not. Stats are
> zone init allocated delta
> private 1000000 918080 81920
> shared 9999000000 9999000000 0.
> Fragment 0:0
> [Error Id: 90ced8b1-b6db-438f-b193-b7634de31b81 on centos-03.qa.lab:31010]
> at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:522) ~[drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:315) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:283) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> Caused by: java.lang.IllegalStateException: Failure while closing accountor. Expected private and shared pools to be set to initial values. However, one or more were not. Stats are
> zone init allocated delta
> private 1000000 918080 81920
> shared 9999000000 9999000000 0.
> at org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200) ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386) ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:325) ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.ops.OperatorContextImpl.close(OperatorContextImpl.java:116) ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.ops.FragmentContext.suppressingClose(FragmentContext.java:405) ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:394) ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:340) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:278) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> ... 4 common frames omitted
> 2015-05-15 00:59:01,963 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO o.a.drill.exec.work.foreman.Foreman - State change requested. CANCELLATION_REQUESTED --> FAILED
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: java.lang.IllegalStateException: Failure while closing accountor. Expected private and shared pools to be set to initial values. However, one or more were not. Stats are
> zone init allocated delta
> private 1000000 918080 81920
> shared 9999000000 9999000000 0.
> Fragment 0:0
> [Error Id: 90ced8b1-b6db-438f-b193-b7634de31b81 on centos-03.qa.lab:31010]
> at org.apache.drill.exec.work.foreman.QueryManager$1.statusUpdate(QueryManager.java:481) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:461) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:90) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:86) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:319) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:283) [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)