You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@impala.apache.org by "Abhishek Rawat (JIRA)" <ji...@apache.org> on 2019/05/21 21:08:00 UTC

[jira] [Resolved] (IMPALA-8537) Negative values reported for tmp-file-mgr.scratch-space-bytes-used under heavy spilling load

     [ https://issues.apache.org/jira/browse/IMPALA-8537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Abhishek Rawat resolved IMPALA-8537.
------------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 3.3.0

> Negative values reported for tmp-file-mgr.scratch-space-bytes-used under heavy spilling load
> --------------------------------------------------------------------------------------------
>
>                 Key: IMPALA-8537
>                 URL: https://issues.apache.org/jira/browse/IMPALA-8537
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 3.3.0
>            Reporter: David Rorke
>            Assignee: Abhishek Rawat
>            Priority: Major
>             Fix For: Impala 3.3.0
>
>         Attachments: bad_spill_metrics.json
>
>
> I'm running a workload that does a lot of spilling and noticed the value reported for tmp-file-mgr.scratch-space-bytes-used is negative on all nodes.
> Some details of the workload and cluster configuration:
>  * Generating a 10 TB TPC-DS partitioned parquet data set (very large sort).
>  * 30 impalads, each with 48 GB RAM and 14 scratch directories (each on a separate drive)
>  * Rough estimate (based on query metrics) of total cumulative aggregate memory spilled across the cluster since restart is 6.5 TB.
> Snapshot of the bad metrics attached.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)