You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2019/07/11 15:50:00 UTC
[jira] [Commented] (KYLIN-4005) Saving Cube of a aggregation
Groups(40 Dimensions, Max Dimension Combination:5) may cause kylin server
OOM
[ https://issues.apache.org/jira/browse/KYLIN-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883115#comment-16883115 ]
ASF subversion and git services commented on KYLIN-4005:
--------------------------------------------------------
Commit 47cb8de4a7baf7982c9a1cc26ba9c4ce68e527ff in kylin's branch refs/heads/master from Liu Shaohui
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=47cb8de ]
KYLIN-4005: Saving Cube of a aggregation Groups(40 Dimensions, Max Dimension Combination:5) may cause kylin server OOM
> Saving Cube of a aggregation Groups(40 Dimensions, Max Dimension Combination:5) may cause kylin server OOM
> ----------------------------------------------------------------------------------------------------------
>
> Key: KYLIN-4005
> URL: https://issues.apache.org/jira/browse/KYLIN-4005
> Project: Kylin
> Issue Type: Bug
> Components: REST Service
> Affects Versions: v2.5.2
> Reporter: Liu Shaohui
> Assignee: Liu Shaohui
> Priority: Critical
> Fix For: Future
>
>
> A user try to save a cube with a aggregation Groups(40 Dimensions, Max Dimension Combination:5) caused the kylin server OOM. The reason is that the DefaultCuboidScheduler will cost a lot memory when calculating all cube ids. The stack is following
> {code}
> http-bio-7070-exec-35
> at java.lang.OutOfMemoryError.<init>()V (OutOfMemoryError.java:48)
> at java.util.HashMap.resize()[Ljava/util/HashMap$Node; (HashMap.java:704)
> at java.util.HashMap.putVal(ILjava/lang/Object;Ljava/lang/Object;ZZ)Ljava/lang/Object; (HashMap.java:663)
> at java.util.HashMap.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (HashMap.java:612)
> at java.util.HashSet.add(Ljava/lang/Object;)Z (HashSet.java:220)
> at java.util.AbstractCollection.addAll(Ljava/util/Collection;)Z (AbstractCollection.java:344)
> at org.apache.kylin.cube.cuboid.DefaultCuboidScheduler.getOnTreeParentsByLayer(Ljava/util/Collection;)Ljava/util/Set; (DefaultCuboidScheduler.java:240)
> at org.apache.kylin.cube.cuboid.DefaultCuboidScheduler.buildTreeBottomUp()Lorg/apache/kylin/common/util/Pair; (DefaultCuboidScheduler.java:183)
> at org.apache.kylin.cube.cuboid.DefaultCuboidScheduler.<init>(Lorg/apache/kylin/cube/model/CubeDesc;)V (DefaultCuboidScheduler.java:58)
> at sun.reflect.GeneratedConstructorAccessor140.newInstance([Ljava/lang/Object;)Ljava/lang/Object; (Unknown Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance([Ljava/lang/Object;)Ljava/lang/Object; (DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance([Ljava/lang/Object;)Ljava/lang/Object; (Constructor.java:423)
> at org.apache.kylin.cube.cuboid.CuboidScheduler.getInstance(Lorg/apache/kylin/cube/model/CubeDesc;)Lorg/apache/kylin/cube/cuboid/CuboidScheduler; (CuboidScheduler.java:41)
> at org.apache.kylin.cube.model.CubeDesc.getInitialCuboidScheduler()Lorg/apache/kylin/cube/cuboid/CuboidScheduler; (CubeDesc.java:750)
> at org.apache.kylin.cube.cuboid.CuboidCLI.simulateCuboidGeneration(Lorg/apache/kylin/cube/model/CubeDesc;Z)I (CuboidCLI.java:47)
> at org.apache.kylin.rest.service.CubeService.updateCubeAndDesc(Lorg/apache/kylin/cube/CubeInstance;Lorg/apache/kylin/cube/model/CubeDesc;Ljava/lang/String;Z)Lorg/apache/kylin/cube/model/CubeDesc; (CubeService.java:287)
> at org.apache.kylin.rest.service.CubeService$$FastClassBySpringCGLIB$$17a07c0e.invoke(ILjava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (Unknown Source)
> at org.springframework.cglib.proxy.MethodProxy.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (MethodProxy.java:204)
> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;Lorg/springframework/cglib/proxy/MethodProxy;)Ljava/lang/Object; (CglibAopProxy.java:669)
> at org.apache.kylin.rest.service.CubeService$$EnhancerBySpringCGLIB$$34de75c4.updateCubeAndDesc(Lorg/apache/kylin/cube/CubeInstance;Lorg/apache/kylin/cube/model/CubeDesc;Ljava/lang/String;Z)Lorg/apache/kylin/cube/model/CubeDesc; (Unknown Source)
> at org.apache.kylin.rest.controller.CubeController.updateCubeDesc(Lorg/apache/kylin/rest/request/CubeRequest;)Lorg/apache/kylin/rest/request/CubeReq
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)