You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@drill.apache.org by "Abhishek Girish (JIRA)" <ji...@apache.org> on 2015/10/08 01:40:27 UTC

[jira] [Created] (DRILL-3913) Possible memory leak during CTAS using 30 TB TPC-H dataset

Abhishek Girish created DRILL-3913:
--------------------------------------

             Summary: Possible memory leak during CTAS using 30 TB TPC-H dataset
                 Key: DRILL-3913
                 URL: https://issues.apache.org/jira/browse/DRILL-3913
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 1.2.0
         Environment: 47 nodes configured with 32 GB Drill Direct memory
            Reporter: Abhishek Girish


8 CTAS queries were executed sequentially to write TPC-H text data into Parquet. After successfully writing a few tables, CTAS failed with OOM.

Restarting Drillbits fixed the problem and re-run of pending CTAS queries completed. This process was done twice in-order to complete all 8 tables to be written. Overall source was 30TB in size. 

Queries attached. Query profile for one of the CTAS which failed is attached. Logs indicated that the Drillbit was out of Direct Memory. 

Can share more details as required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)