You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "hongbin ma (JIRA)" <ji...@apache.org> on 2016/07/01 10:52:11 UTC
[jira] [Commented] (KYLIN-1837) Feature request - cross cube reuse
of Kylin fact/lookup snapshots ...
[ https://issues.apache.org/jira/browse/KYLIN-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358780#comment-15358780 ]
hongbin ma commented on KYLIN-1837:
-----------------------------------
have you checked https://issues.apache.org/jira/browse/KYLIN-1388, does it cover your problem?
> Feature request - cross cube reuse of Kylin fact/lookup snapshots ...
> ---------------------------------------------------------------------
>
> Key: KYLIN-1837
> URL: https://issues.apache.org/jira/browse/KYLIN-1837
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Affects Versions: all
> Reporter: Richard Calaba
> Assignee: Dong Li
>
> Hello Kylin gurus,
> while debugging some issues with high cardinality dimensions - which obviously requires large data to be processed to emulate the problem thus the Cube Build process takes significant time ... I came to this idea:
> - Cannot be the Snapshot logic - be resued cross cubes ??
> - Let's say I have cube 1 and cube 2 which is clone of cube 1 maybe with removed some dimnesions or even having same dimensions and just having different measures definition ...
> - Cube 1 build fails somewhere in later steps (snaphost already built) in step 1 I believe
> - Running build of 2nd cube - which let's say is using exactly same dimensions table and in fact also same fact table - this also requires long run because in the Step 1 the build process is calculating the snaphots ... which are already calculated (and still not discared) by the Build Job of Cube 1 ....
> Is there any chance to define some snapshots reuse scenarios like that (same model/DB tables referred) ... so the modelling &build time can be shortened while playing with the cube design ??? (i.e. testing various optimizations like joint dimensions, etc ...- those should not be impacted by the source data stored in the alread calculated snapshots, right ?
> Obviously that should be an option while scheduling Cube Build to enable/disable reuse of snapshots from other similar cubes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)