You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "kangkaisen (JIRA)" <ji...@apache.org> on 2017/04/14 09:00:49 UTC
[jira] [Updated] (KYLIN-2547) Fix the bug of multi-process
concurrence in mergeCubeSegment
[ https://issues.apache.org/jira/browse/KYLIN-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
kangkaisen updated KYLIN-2547:
------------------------------
Description:
There is a minor bug in "Update Cube Info" step when build a cube in distributed env, which will make this job failed, but the next auto merge job won't be affected and will succeed. So the cube data still is consistent and correct.
{code:java}
Caused by: java.lang.IllegalStateException: Segments overlap: waimai_dolphin_topic_flow_activity_expose_food_d_cube[20170405000000_20170412000000] and waimai_dolphin_topic_flow_activity_expose_food_d_cube[20170405000000_20170412000000]
at org.apache.kylin.cube.CubeValidator.validate(CubeValidator.java:85)
at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:359)
at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:386)
at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:302)
at org.apache.kylin.cube.CubeManager.mergeSegments(CubeManager.java:533)
at org.apache.kylin.rest.service.CubeService.mergeCubeSegment(CubeService.java:635)
at org.apache.kylin.rest.service.CubeService.updateOnNewSegmentReady(CubeService.java:587)
at org.apache.kylin.rest.service.CubeServiceFastClassBySpringCGLIB17a07c0e.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxyDynamicAdvisedInterceptor.intercept(CglibAopProxy.java:629)
at org.apache.kylin.rest.service.CubeServiceEnhancerBySpringCGLIB$$c6fabb3f.updateOnNewSegmentReady()
at org.apache.kylin.rest.service.CacheService.rebuildCubeCache(CacheService.java:237)
at org.apache.kylin.rest.service.CacheService.access$000(CacheService.java:62)
at org.apache.kylin.rest.service.CacheService$1.afterCubeUpdate(CacheService.java:86)
at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:305)
at org.apache.kylin.cube.CubeManager.promoteNewlyBuiltSegments(CubeManager.java:735)
at org.apache.kylin.engine.mr.steps.UpdateCubeInfoAfterBuildStep.doWork(UpdateCubeInfoAfterBuildStep.java:62)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
... 6 more
{code}
was:
There is a minor bug in "Update Cube Info" when build a cube and in distributed env.
{code:java}
Caused by: java.lang.IllegalStateException: Segments overlap: waimai_dolphin_topic_flow_activity_expose_food_d_cube[20170405000000_20170412000000] and waimai_dolphin_topic_flow_activity_expose_food_d_cube[20170405000000_20170412000000]
at org.apache.kylin.cube.CubeValidator.validate(CubeValidator.java:85)
at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:359)
at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:386)
at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:302)
at org.apache.kylin.cube.CubeManager.mergeSegments(CubeManager.java:533)
at org.apache.kylin.rest.service.CubeService.mergeCubeSegment(CubeService.java:635)
at org.apache.kylin.rest.service.CubeService.updateOnNewSegmentReady(CubeService.java:587)
at org.apache.kylin.rest.service.CubeServiceFastClassBySpringCGLIB17a07c0e.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxyDynamicAdvisedInterceptor.intercept(CglibAopProxy.java:629)
at org.apache.kylin.rest.service.CubeServiceEnhancerBySpringCGLIB$$c6fabb3f.updateOnNewSegmentReady()
at org.apache.kylin.rest.service.CacheService.rebuildCubeCache(CacheService.java:237)
at org.apache.kylin.rest.service.CacheService.access$000(CacheService.java:62)
at org.apache.kylin.rest.service.CacheService$1.afterCubeUpdate(CacheService.java:86)
at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:305)
at org.apache.kylin.cube.CubeManager.promoteNewlyBuiltSegments(CubeManager.java:735)
at org.apache.kylin.engine.mr.steps.UpdateCubeInfoAfterBuildStep.doWork(UpdateCubeInfoAfterBuildStep.java:62)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
... 6 more
{code}
> Fix the bug of multi-process concurrence in mergeCubeSegment
> ------------------------------------------------------------
>
> Key: KYLIN-2547
> URL: https://issues.apache.org/jira/browse/KYLIN-2547
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
> Affects Versions: v2.0.0
> Reporter: kangkaisen
> Assignee: kangkaisen
> Priority: Minor
> Fix For: v2.0.0
>
>
> There is a minor bug in "Update Cube Info" step when build a cube in distributed env, which will make this job failed, but the next auto merge job won't be affected and will succeed. So the cube data still is consistent and correct.
> {code:java}
> Caused by: java.lang.IllegalStateException: Segments overlap: waimai_dolphin_topic_flow_activity_expose_food_d_cube[20170405000000_20170412000000] and waimai_dolphin_topic_flow_activity_expose_food_d_cube[20170405000000_20170412000000]
> at org.apache.kylin.cube.CubeValidator.validate(CubeValidator.java:85)
> at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:359)
> at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:386)
> at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:302)
> at org.apache.kylin.cube.CubeManager.mergeSegments(CubeManager.java:533)
> at org.apache.kylin.rest.service.CubeService.mergeCubeSegment(CubeService.java:635)
> at org.apache.kylin.rest.service.CubeService.updateOnNewSegmentReady(CubeService.java:587)
> at org.apache.kylin.rest.service.CubeServiceFastClassBySpringCGLIB17a07c0e.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at org.springframework.aop.framework.CglibAopProxyDynamicAdvisedInterceptor.intercept(CglibAopProxy.java:629)
> at org.apache.kylin.rest.service.CubeServiceEnhancerBySpringCGLIB$$c6fabb3f.updateOnNewSegmentReady()
> at org.apache.kylin.rest.service.CacheService.rebuildCubeCache(CacheService.java:237)
> at org.apache.kylin.rest.service.CacheService.access$000(CacheService.java:62)
> at org.apache.kylin.rest.service.CacheService$1.afterCubeUpdate(CacheService.java:86)
> at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:305)
> at org.apache.kylin.cube.CubeManager.promoteNewlyBuiltSegments(CubeManager.java:735)
> at org.apache.kylin.engine.mr.steps.UpdateCubeInfoAfterBuildStep.doWork(UpdateCubeInfoAfterBuildStep.java:62)
> at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
> ... 6 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)