You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@impala.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2018/08/15 19:25:00 UTC

[jira] [Created] (IMPALA-7453) Intern HdfsStorageDescriptors

Todd Lipcon created IMPALA-7453:
-----------------------------------

             Summary: Intern HdfsStorageDescriptors
                 Key: IMPALA-7453
                 URL: https://issues.apache.org/jira/browse/IMPALA-7453
             Project: IMPALA
          Issue Type: Improvement
          Components: Catalog
            Reporter: Todd Lipcon


Every partition currently has an HdfsStorageDescriptor attached. In most cases, the number of unique storage descriptors in a warehouse is pretty low (most partitions use the same escaping, file formats, etc). For example, in the functional test data load, we only have 24 unique SDs across ~10k partitions. Each object takes 32 bytes (with compressed oops) or 40 (without). So, we can get some small memory/object-count savings by interning these.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)