You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/06/07 20:35:00 UTC
[jira] [Commented] (DRILL-6477) Drillbit hangs/crashes with OOME
Java Heap Space for a large query through WebUI
[ https://issues.apache.org/jira/browse/DRILL-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505224#comment-16505224 ]
ASF GitHub Bot commented on DRILL-6477:
---------------------------------------
kkhatua opened a new pull request #1309: DRILL-6477: Drillbit crashes with OOME (Heap) for a large WebUI query
URL: https://github.com/apache/drill/pull/1309
For queries submitted through the WebUI and retrieving a large result-set, the Drillbit often hangs or crashes due to the (foreman) Drillbit running out of Heap memory.
This is because the Web client translates the result set into a massive object in the heap-space and tries to send that back to the browser. This results in the VM thread actively trying to perform GC if the memory is not sufficient.
The workaround is to have the active webConnection of the query periodically timeout to allow for checking the consumed heap-space. A level of 0.85 (i.e. 85%) is set as default threshold, crossing which, a query submitted through the REST API is marked and failed.
In addition, a user exception is thrown, indicting the cause of the query failing, along with alternative suggestions for re-executing the query.
This is the example of a query that tried to scan an entire 60M row table through the browser. The query failed with the following error message:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Drillbit hangs/crashes with OOME Java Heap Space for a large query through WebUI
> --------------------------------------------------------------------------------
>
> Key: DRILL-6477
> URL: https://issues.apache.org/jira/browse/DRILL-6477
> Project: Apache Drill
> Issue Type: Bug
> Components: Web Server
> Affects Versions: 1.13.0
> Reporter: Kunal Khatua
> Assignee: Kunal Khatua
> Priority: Major
> Fix For: 1.14.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> For queries submitted through the WebUI and retrieving a large resultset, the Drillbit often hangs or crashes due to the (foreman) Drillbit running out of Heap memory.
> This is because the Web client translates the resultset into a massive object in the heap-space and tries to send that back to the browser. This results in the VM thread actively trying to perform GC if the memory is not sufficient.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)