You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "lichenglin (JIRA)" <ji...@apache.org> on 2016/03/15 13:35:33 UTC
[jira] [Updated] (SPARK-13907) Imporvement the cube with the Fast
Cubing In apache Kylin
[ https://issues.apache.org/jira/browse/SPARK-13907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
lichenglin updated SPARK-13907:
-------------------------------
Description:
I tried to build a cube on a 100 million data set.
When I set 9 fields to build the cube with 10 cores.
It nearly coast me a whole day to finish the job.
At the same time, it generate almost 1”TB“ data in the "/tmp“ folder.
Could we refer to the ”fast cube“ algorithm in apache Kylin
To make the cube builder more quickly???
For example
group(A,B,C)'s result can be create by the groupdata(A,B)
was:
I tried to build a cube on a 100 million data set.
When I set 9 fields to build the cube with 10 cores.
It nearly coast me a whole day to finish the job.
At the same time, it generate almost 1”TB“ data in the "/tmp“ folder.
Could we refer to the ”fast cube“ algorithm in apache Kylin
To make the cube builder more quickly???
> Imporvement the cube with the Fast Cubing In apache Kylin
> ---------------------------------------------------------
>
> Key: SPARK-13907
> URL: https://issues.apache.org/jira/browse/SPARK-13907
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 1.6.1
> Reporter: lichenglin
>
> I tried to build a cube on a 100 million data set.
> When I set 9 fields to build the cube with 10 cores.
> It nearly coast me a whole day to finish the job.
> At the same time, it generate almost 1”TB“ data in the "/tmp“ folder.
> Could we refer to the ”fast cube“ algorithm in apache Kylin
> To make the cube builder more quickly???
> For example
> group(A,B,C)'s result can be create by the groupdata(A,B)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org