You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues-all@impala.apache.org by "Alexander Saydakov (Jira)" <ji...@apache.org> on 2021/10/07 21:48:00 UTC

[jira] [Created] (IMPALA-10956) datasketches UDFS: memory leak and merge overhead

Alexander Saydakov created IMPALA-10956:
-------------------------------------------

             Summary: datasketches UDFS: memory leak and merge overhead
                 Key: IMPALA-10956
                 URL: https://issues.apache.org/jira/browse/IMPALA-10956
             Project: IMPALA
          Issue Type: Bug
          Components: Backend
    Affects Versions: Impala 3.4.0
            Reporter: Alexander Saydakov


I believe that there are memory leaks in aggregate UDFs that are using Apache Datasketches. Sketch or union objects are placed in a buffer to maintain the state of the aggregation, and later deallocated without calling destructors for those objects.

Also there is unnecessary overhead during the merge phase of some of those UDFs, when an instance of a union is created for every merge operation and finalized by calling a quite expensive get_result() every time as well. This second issue is about performance, not a bug.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org