[jira] [Closed] (DRILL-4542) if external sort fails to spill to disk, memory is leaked and wrong error message is displayed

["Boaz Ben-Zvi (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DRILL-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Boaz Ben-Zvi closed DRILL-4542.
-------------------------------
       Resolution: Duplicate
    Fix Version/s: 1.9.0

Exactly the same as DRILL-3898 

> if external sort fails to spill to disk, memory is leaked and wrong error message is displayed
> ----------------------------------------------------------------------------------------------
>
>                 Key: DRILL-4542
>                 URL: https://issues.apache.org/jira/browse/DRILL-4542
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Relational Operators
>    Affects Versions: 1.6.0
>         Environment: When external sort fails spilling to disk, for example because there is no space left on disk, the query fails with an error like this:
> {noformat}
> SYSTEM ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: (376832)
> Allocator(op:1:0:6:ExternalSort) 20000000/376832/408432256/429496729 (res/actual/peak/limit)
> {noformat}
> 2 problems here:
> 1. we are leaking memory
> 2. the "real" problem is not shown in the error message
> Looking at the logs we can find the real issue, generally right after the previous error (edited for the sake of clarity):
> {noformat}
> 2016-03-26 02:22:28,576 [drill-executor-119] ERROR o.a.d.exec.server.BootStrapContext - org.apache.drill.exec.work.WorkManager$WorkerBee$2.run() leaked an exception.
> org.apache.hadoop.fs.FSError: java.io.IOException: No space left on device
>         at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:249) ~[hadoop-common-2.7.0-mapr-1602.jar:na]
> 		[...]
>         at org.apache.drill.exec.cache.VectorAccessibleSerializable.writeToStream(VectorAccessibleSerializable.java:172) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.physical.impl.xsort.BatchGroup.addBatch(BatchGroup.java:96) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.mergeAndSpill(ExternalSortBatch.java:562) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:394) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> 		[...]
> {noformat}
>            Reporter: Deneche A. Hakim
>            Assignee: Boaz Ben-Zvi
>             Fix For: 1.9.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)