You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by "Sean Hsuan-Yi Chu (JIRA)" <ji...@apache.org> on 2015/05/14 19:47:00 UTC
[jira] [Created] (DRILL-3085) In ExternalSortBatch, Memory Leak in
Runtime Generation Code
Sean Hsuan-Yi Chu created DRILL-3085:
----------------------------------------
Summary: In ExternalSortBatch, Memory Leak in Runtime Generation Code
Key: DRILL-3085
URL: https://issues.apache.org/jira/browse/DRILL-3085
Project: Apache Drill
Issue Type: Bug
Components: Execution - Codegen
Reporter: Sean Hsuan-Yi Chu
Assignee: Sean Hsuan-Yi Chu
Priority: Critical
This case is related to DRILL-3065.
In ExternalSortBatch, we have an MSorter to do sorting thing. In this class, there are two SelectionVector4, vector4 and aux. If we fail at the time just after either gets new memory, the close() method would fail to clean their allocated memory properly.
To reproduce this problem, inject an exception at the last step of MSortTemplate.setup()
Detailed Information:
1. Query:
select n_name from cp.`tpch/nation.parquet` order by n_name
*. Using this query alone cannot help reproduce the issue. We still need to inject the exception at the right place.
2. Data:
cp.`tpch/nation.parquet`
3. Log:
java.lang.IllegalStateException: Failure while closing accountor. Expected private and shared pools to be set to initial values. However, one or more were not. Stats are
zone init allocated delta
private 0 0 0
shared 3221225472 3195686243 25539229.
at org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200)
at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386)
at org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:175)
at org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75)
at com.google.common.io.Closeables.close(Closeables.java:77)
at com.google.common.io.Closeables.closeQuietly(Closeables.java:108)
at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:292)
at org.apache.drill.BaseTestQuery.closeClient(BaseTestQuery.java:238)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Exception in thread "Drillbit-ShutdownHook#0" java.lang.RuntimeException: Caught exception closing Drillbit started from
org.apache.drill.common.StackTrace.<init>:36
org.apache.drill.exec.server.Drillbit.run:250
org.apache.drill.BaseTestQuery.openClient:180
org.apache.drill.BaseTestQuery.setupDefaultTestCluster:116
sun.reflect.NativeMethodAccessorImpl.invoke0:-2
sun.reflect.NativeMethodAccessorImpl.invoke:57
sun.reflect.DelegatingMethodAccessorImpl.invoke:43
java.lang.reflect.Method.invoke:606
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall:47
org.junit.internal.runners.model.ReflectiveCallable.run:12
org.junit.runners.model.FrameworkMethod.invokeExplosively:44
org.junit.internal.runners.statements.RunBefores.evaluate:24
org.junit.internal.runners.statements.RunAfters.evaluate:27
org.junit.runners.ParentRunner.run:309
org.junit.runner.JUnitCore.run:160
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs:74
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart:211
com.intellij.rt.execution.junit.JUnitStarter.main:67
sun.reflect.NativeMethodAccessorImpl.invoke0:-2
sun.reflect.NativeMethodAccessorImpl.invoke:57
sun.reflect.DelegatingMethodAccessorImpl.invoke:43
java.lang.reflect.Method.invoke:606
at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:333)
Caused by: java.lang.IllegalStateException: Failure while closing accountor. Expected private and shared pools to be set to initial values. However, one or more were not. Stats are
zone init allocated delta
private 0 0 0
shared 3221225472 3195686243 25539229.
at org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200)
at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386)
at org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:175)
at org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75)
at com.google.common.io.Closeables.close(Closeables.java:77)
at com.google.common.io.Closeables.closeQuietly(Closeables.java:108)
at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:292)
at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:330)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)