You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by ShaoFeng Shi <sh...@gmail.com> on 2017/03/03 10:05:10 UTC
Re: Which version of spark is kylin 2.0 Beta using ?—— How can others get data used in your test？
hi mars,
The dataset can be download from internet, you can refer this article to get it: http://kyligence.io/2016/11/1729/
For the cube model, I can share it later (next week) when back to office.
Get Outlook for iOS
On Thu, Mar 2, 2017 at 4:53 PM +0700, "Mars Xu" <xu...@gmail.com> wrote:
Hello Shaofeng,
Thanks a lot for providing the by-layer-spark cubing model, I have seen the blog which metioned your test on Spark, My question is can other people get the test data ? If many users can use the same data to test kylin 2.0, may be it will improve kylin more efficiently.
Thanks again.
Mars.
在 2017年3月1日，下午5:27，ShaoFeng Shi <sh...@apache.org> 写道：
Kylin embed a spark, which is spark v1.6.3; I added a blog to briefly introduce this, later will add a doc to instruct on how to config:
https://kylin.apache.org/blog/2017/02/23/by-layer-spark-cubing/
To customize the spark conf, please do that in kylin.properties with prefix "kylin.engine.spark-conf."
2017-03-01 15:49 GMT+08:00 Mars Xu <xu...@gmail.com>:
Hi ,
I have install kylin 2.0 beta on my cluster which has already deploy a spark 2.1, but when I startup kylin server and build cube, the Web UI of Cube Build Step show it is spark 1.6.3.
Is that kylin already include a spark version ? or some other else I need to configure ?
Thanks
Mars.
--
Best regards,
Shaofeng Shi 史少锋
Re: Which version of spark is kylin 2.0 Beta using ?—— How can others get data used in your test？
Posted by Mars Xu <xu...@gmail.com>.
Get it, Thanks a lot.
> 在 2017年3月6日，上午9:50，ShaoFeng Shi <sh...@gmail.com> 写道：
>
> Hi Mars,
>
> The cube "airline_cube1" is the cube using spark engine; others are using MR, their logical structure is the same; Please note the properties "kylin.engine.spark.config-override.*" in conf/kylin.properties are renamed to "kylin.engine.spark-conf.*" in the beta release. For your reference.
>
> 2017-03-03 18:05 GMT+08:00 ShaoFeng Shi <shaofengshi@gmail.com <ma...@gmail.com>>:
> hi mars,
>
> The dataset can be download from internet, you can refer this article to get it: http://kyligence.io/2016/11/1729/ <http://kyligence.io/2016/11/1729/>
>
> For the cube model, I can share it later (next week) when back to office.
>
>
>
> Get Outlook for iOS <https://aka.ms/o0ukef>
>
>
>
> On Thu, Mar 2, 2017 at 4:53 PM +0700, "Mars Xu" <xujiao.mycafe@gmail.com <ma...@gmail.com>> wrote:
>
> Hello Shaofeng,
>
> Thanks a lot for providing the by-layer-spark cubing model, I have seen the blog which metioned your test on Spark, My question is can other people get the test data ?
> <PastedGraphic-1.png>
> If many users can use the same data to test kylin 2.0, may be it will improve kylin more efficiently.
>
> Thanks again.
>
> Mars.
>
>
>
>> 在 2017年3月1日，下午5:27，ShaoFeng Shi <shaofengshi@apache.org <ma...@apache.org>> 写道：
>>
>> Kylin embed a spark, which is spark v1.6.3; I added a blog to briefly introduce this, later will add a doc to instruct on how to config:
>>
>> https://kylin.apache.org/blog/2017/02/23/by-layer-spark-cubing/ <https://kylin.apache.org/blog/2017/02/23/by-layer-spark-cubing/>
>>
>> To customize the spark conf, please do that in kylin.properties with prefix "kylin.engine.spark-conf."
>>
>> 2017-03-01 15:49 GMT+08:00 Mars Xu <xujiao.mycafe@gmail.com <ma...@gmail.com>>:
>> Hi ,
>> I have install kylin 2.0 beta on my cluster which has already deploy a spark 2.1, but when I startup kylin server and build cube, the Web UI of Cube Build Step show it is spark 1.6.3.
>>
>> Is that kylin already include a spark version ? or some other else I need to configure ?
>>
>>
>> Thanks
>>
>> Mars.
>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>>
>
>
> <project_2016_10_10_09_00_37.zip>
Re: Which version of spark is kylin 2.0 Beta using ?—— How can others get data used in your test？
Posted by ShaoFeng Shi <sh...@gmail.com>.
Hi Mars,
The cube "airline_cube1" is the cube using spark engine; others are using
MR, their logical structure is the same; Please note the properties
"kylin.engine.spark.config-override.*" in conf/kylin.properties are renamed
to "kylin.engine.spark-conf.*" in the beta release. For your reference.
2017-03-03 18:05 GMT+08:00 ShaoFeng Shi <sh...@gmail.com>:
> hi mars,
>
> The dataset can be download from internet, you can refer this article to
> get it: http://kyligence.io/2016/11/1729/
>
> For the cube model, I can share it later (next week) when back to office.
>
>
>
> Get Outlook for iOS <https://aka.ms/o0ukef>
>
>
>
>
> On Thu, Mar 2, 2017 at 4:53 PM +0700, "Mars Xu" <xu...@gmail.com>
> wrote:
>
> Hello Shaofeng,
>>
>> Thanks a lot for providing the by-layer-spark cubing model, I have
>> seen the blog which metioned your test on Spark, My question is can other
>> people get the test data ?
>> If many users can use the same data to test kylin 2.0, may be it will
>> improve kylin more efficiently.
>>
>> Thanks again.
>>
>> Mars.
>>
>>
>>
>> 在 2017年3月1日，下午5:27，ShaoFeng Shi <sh...@apache.org> 写道：
>>
>> Kylin embed a spark, which is spark v1.6.3; I added a blog to briefly
>> introduce this, later will add a doc to instruct on how to config:
>>
>> https://kylin.apache.org/blog/2017/02/23/by-layer-spark-cubing/
>>
>> To customize the spark conf, please do that in kylin.properties with
>> prefix "kylin.engine.spark-conf."
>>
>> 2017-03-01 15:49 GMT+08:00 Mars Xu <xu...@gmail.com>:
>>
>>> Hi ,
>>> I have install kylin 2.0 beta on my cluster which has already deploy
>>> a spark 2.1, but when I startup kylin server and build cube, the Web UI of
>>> Cube Build Step show it is spark 1.6.3.
>>>
>>> Is that kylin already include a spark version ? or some other else I
>>> need to configure ?
>>>
>>>
>>> Thanks
>>>
>>> Mars.
>>
>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>>
>>
>>
Re: Which version of spark is kylin 2.0 Beta using ?—— How can others get data used in your test？
Posted by ShaoFeng Shi <sh...@gmail.com>.
Hi Mars,
The cube "airline_cube1" is the cube using spark engine; others are using
MR, their logical structure is the same; Please note the properties
"kylin.engine.spark.config-override.*" in conf/kylin.properties are renamed
to "kylin.engine.spark-conf.*" in the beta release. For your reference.
2017-03-03 18:05 GMT+08:00 ShaoFeng Shi <sh...@gmail.com>:
> hi mars,
>
> The dataset can be download from internet, you can refer this article to
> get it: http://kyligence.io/2016/11/1729/
>
> For the cube model, I can share it later (next week) when back to office.
>
>
>
> Get Outlook for iOS <https://aka.ms/o0ukef>
>
>
>
>
> On Thu, Mar 2, 2017 at 4:53 PM +0700, "Mars Xu" <xu...@gmail.com>
> wrote:
>
> Hello Shaofeng,
>>
>> Thanks a lot for providing the by-layer-spark cubing model, I have
>> seen the blog which metioned your test on Spark, My question is can other
>> people get the test data ?
>> If many users can use the same data to test kylin 2.0, may be it will
>> improve kylin more efficiently.
>>
>> Thanks again.
>>
>> Mars.
>>
>>
>>
>> 在 2017年3月1日，下午5:27，ShaoFeng Shi <sh...@apache.org> 写道：
>>
>> Kylin embed a spark, which is spark v1.6.3; I added a blog to briefly
>> introduce this, later will add a doc to instruct on how to config:
>>
>> https://kylin.apache.org/blog/2017/02/23/by-layer-spark-cubing/
>>
>> To customize the spark conf, please do that in kylin.properties with
>> prefix "kylin.engine.spark-conf."
>>
>> 2017-03-01 15:49 GMT+08:00 Mars Xu <xu...@gmail.com>:
>>
>>> Hi ,
>>> I have install kylin 2.0 beta on my cluster which has already deploy
>>> a spark 2.1, but when I startup kylin server and build cube, the Web UI of
>>> Cube Build Step show it is spark 1.6.3.
>>>
>>> Is that kylin already include a spark version ? or some other else I
>>> need to configure ?
>>>
>>>
>>> Thanks
>>>
>>> Mars.
>>
>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>>
>>
>>