You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Matt Keranen (JIRA)" <ji...@apache.org> on 2016/04/18 21:13:25 UTC
[jira] [Comment Edited] (DRILL-4616) Running out of /tmp results in
"Memory was leaked by query"
[ https://issues.apache.org/jira/browse/DRILL-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246328#comment-15246328 ]
Matt Keranen edited comment on DRILL-4616 at 4/18/16 7:12 PM:
--------------------------------------------------------------
This appears to be caused by /tmp on the native filesystem running out of available space:
{noformat}
2016-04-18 20:46:33,633 [drill-executor-13] ERROR o.a.d.exec.server.BootStrapContext - org.apache.drill.exec.work.WorkManager$WorkerBee$1.run() leaked an exception.
org.apache.hadoop.fs.FSError: java.io.IOException: No space left on device
{noformat}
I did enter a tmp option in drill-override.conf which did not appear to be respected:
{noformat}
tmp: {
directories: ["/localdata/drill/tmp"],
filesystem: "drill-local:///"
}
{noformat}
So as a test I symlinked a path to a larger device in /tmp, which is a not a permanent solution.
Is there a better approach to specifying where Drill uses tmp or spill space on disk?
was (Author: mattk):
This could be caused by /tmp on the native filesystem running out of available space:
{noformat}
2016-04-18 20:46:33,633 [drill-executor-13] ERROR o.a.d.exec.server.BootStrapContext - org.apache.drill.exec.work.WorkManager$WorkerBee$1.run() leaked an exception.
org.apache.hadoop.fs.FSError: java.io.IOException: No space left on device
{noformat}
I did enter a tmp option in drill-override.conf which did not appear to be respected:
{noformat}
tmp: {
directories: ["/localdata/drill/tmp"],
filesystem: "drill-local:///"
}
{noformat}
So as a test I symlinked a path to a larger device in /tmp, which is a not a permanent solution.
Is there a better approach to specifying where Drill uses tmp or spill space on disk?
> Running out of /tmp results in "Memory was leaked by query"
> -----------------------------------------------------------
>
> Key: DRILL-4616
> URL: https://issues.apache.org/jira/browse/DRILL-4616
> Project: Apache Drill
> Issue Type: Bug
> Components: Server
> Affects Versions: 1.6.0
> Environment: MapR community cluster v. 5.1.0.37549.GA, 4 nodes Ubuntu 14.04, MapR-FS
> Also Vanilla Hadoop 2.7.0 HDFS
> Each node has Maximum Direct Memory of 25,769,803,776
> Reporter: Matt Keranen
>
> Attempting to convert csv file data to partitioned Parquet files via sqlline with as
> {noformat}
> CREATE TABLE () PARTITION BY (date_tm) AS SELECT ...
> Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: (523264)
> Allocator(op:1:16:5:ExternalSort) 20000000/523264/1361178240/1431655765 (res/actual/peak/limit)
> Fragment 1:16
> [Error Id: cea4a79d-3e85-4e51-b6c2-1539f40dee10 on es05:31010]
> (java.lang.IllegalStateException) Memory was leaked by query. Memory leaked: (523264)
> Allocator(op:1:16:5:ExternalSort) 20000000/523264/1361178240/1431655765 (res/actual/peak/limit)
> org.apache.drill.exec.memory.BaseAllocator.close():492
> org.apache.drill.exec.ops.OperatorContextImpl.close():124
> org.apache.drill.exec.ops.FragmentContext.suppressingClose():416
> org.apache.drill.exec.ops.FragmentContext.close():405
> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources():343
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup():180
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():287
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745 (state=,code=0)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)