You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@orc.apache.org by "Gopal V (JIRA)" <ji...@apache.org> on 2019/05/16 00:31:00 UTC

[jira] [Created] (ORC-500) PPD: StripeStatistics::getColumnStatistics() does not column prune

Gopal V created ORC-500:
---------------------------

             Summary: PPD: StripeStatistics::getColumnStatistics() does not column prune
                 Key: ORC-500
                 URL: https://issues.apache.org/jira/browse/ORC-500
             Project: ORC
          Issue Type: Bug
          Components: Java
    Affects Versions: 1.5.5
            Reporter: Gopal V


StripeStatistics::getColumnStatistics() will deserialize all column statistics, even if the ORC predicate push-down and the eventual SARG applies to a subset.

This is affecting DATE push-down into a very wide table, where the deserialization of the statistics is the host spot during split-generation, due to the new Text() allocation within String statistics implementation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)