You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Ji Qi (Jira)" <ji...@apache.org> on 2022/03/21 05:30:00 UTC

[jira] [Created] (HUDI-3670) SqlQueryBasedTransformer leaks temp views in continuous mode

Ji Qi created HUDI-3670:
---------------------------

             Summary: SqlQueryBasedTransformer leaks temp views in continuous mode
                 Key: HUDI-3670
                 URL: https://issues.apache.org/jira/browse/HUDI-3670
             Project: Apache Hudi
          Issue Type: Bug
          Components: deltastreamer
            Reporter: Ji Qi


In the Sql transformers, a new temp view with a random name is created for each incoming batch, but the temp view is never dropped.

This causes a resource leak in Spark SessionState's Catalog. For long-running deltastreamer jobs, the temp views created causes HiveSessionCatalog to take up a lot of memory.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)