You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@kylin.apache.org by "ZhouKang (Jira)" <ji...@apache.org> on 2019/09/30 07:44:00 UTC

[jira] [Created] (KYLIN-4185) CubeStatsReader estimate wrong cube size

ZhouKang created KYLIN-4185:
-------------------------------

             Summary: CubeStatsReader estimate wrong cube size
                 Key: KYLIN-4185
                 URL: https://issues.apache.org/jira/browse/KYLIN-4185
             Project: Kylin
          Issue Type: Improvement
            Reporter: ZhouKang


CubeStatsReader estimate wrong cube size, which cause a lot of problems.

when the estimated size is much larger than the real size, the spark application's executor number is small, and cube build step will take a long time. sometime the step will failed due to the large dataset.

When the estimated size is much smaller than the real size. the cuboid file in HDFS is small, and there are much of cuboid file.

 

In our production environment, both the two situation happened.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)