You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by ShaoFeng Shi <sh...@apache.org> on 2020/02/26 07:48:27 UTC
[DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Hello, Kylin users and developers,
As we know Hadoop 3 and HBase 2 have released for some time. Kylin starts
to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of HBase 1 and
2 are incompatible, we need to keep different branches for them. And in
each release, we need to build separate packages and do a round of testing
for them separately. Furthermore, Cloudera's API difference with the Apache
release makes the situation worse; We need to build 4 binary packages for
reach release. That has spent much of our manual effort and computing
resources.
Today, Hadoop 3 + HBase 2 becomes enough mature and stable for production
use; And we see more and more users are starting to use the new versions.
We think it is time for Kylin to totally upgrade to the new version. So
that we can focus more on Kylin itself, instead of environments.
Here is my proposal:
1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a close
version);
2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous releases;
3) We will re-evaluate the need for building binary packages for Cloudera
release. (we may raise another discuss)
Please let us know your comments. And please also understand with the
limited resource we couldn't support multiple Hadoop versions...
Thanks!
Best regards,
Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofengshi@apache.org
Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscribe@kylin.apache.org
Join Kylin dev mail group: dev-subscribe@kylin.apache.org
回复： [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Posted by 猪兜兜 <12...@qq.com>.
恻然
------------------ 原始邮件 ------------------
发件人: "Luke Han"<luke.hq@gmail.com>;
发送时间: 2020年3月5日(星期四) 下午3:49
收件人: "user"<user@kylin.apache.org>;
抄送: "dev"<dev@kylin.apache.org>;
主题: Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
+1 the new Kylin version should target Hadoop 3
Best Regards!
---------------------
Luke Han
On Mon, Mar 2, 2020 at 4:41 PM Billy Liu <billyliu@apache.org> wrote:
> +1. Let's move to Hadoop3
>
> With Warm regards
>
> Billy Liu
>
>
> ShaoFeng Shi <shaofengshi@apache.org> 于2020年2月27日周四 下午10:07写道：
>
>> Hi Yang,
>>
>> The main difference between 2.6 and 3.0 is the new real-time OLAP
>> feature. Hadoop 2 users can select either of them, depends on whether they
>> need the real-time feature.
>>
>> After 3.0, the next major features would be the Flink cube engine
>> (planned in v3.1) and the Parquet storage (early stage, maybe in v4.0).
>>
>> When the parquet storage is released, as the dependency on HBase can be
>> dropped, then we assume the API issue will easier than today. We can
>> re-evaluate the possibility to support Hadoop 2.
>>
>> So I think the impact on today's Hadoop 2 users is acceptable. Not
>> mention that they still can manually compile that.
>>
>>
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>> Apache Kylin PMC
>> Email: shaofengshi@apache.org
>>
>> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
>> Join Kylin user mail group: user-subscribe@kylin.apache.org
>> Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>>
>>
>>
>>
>> Li Yang <liyang@apache.org> 于2020年2月27日周四 上午7:37写道：
>>
>>> The proposal means Kylin 3.0 will be the last major version that supports
>>> Hadoop 2.
>>>
>>> What will be recommended version for Hadoop 2 users after this? I feel
>>> the
>>> latest stable version of 2.6 is better than 3.0.
>>>
>>> Anyway, I'm fine with moving focus to Hadoop 3. That is the direction.
>>> However we shall also think about what it means for Hadoop 2 users.
>>> Questions like below shall also be answered.
>>>
>>> - What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not
>>> sound right here.)
>>> - How that version/branch will be maintained?
>>>
>>> +1 in general
>>>
>>> Regards
>>> -Yang
>>>
>>>
>>> On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang <zhoukangcn@outlook.com>
>>> wrote:
>>>
>>> > +1
>>> >
>>> >
>>> > > 2020年2月26日 下午3:48，ShaoFeng Shi <shaofengshi@apache.org> 写道：
>>> > >
>>> > > Hello, Kylin users and developers,
>>> > >
>>> > > As we know Hadoop 3 and HBase 2 have released for some time. Kylin
>>> > starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of
>>> HBase
>>> > 1 and 2 are incompatible, we need to keep different branches for them.
>>> And
>>> > in each release, we need to build separate packages and do a round of
>>> > testing for them separately. Furthermore, Cloudera's API difference
>>> with
>>> > the Apache release makes the situation worse; We need to build 4 binary
>>> > packages for reach release. That has spent much of our manual effort
>>> and
>>> > computing resources.
>>> > >
>>> > > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for
>>> > production use; And we see more and more users are starting to use the
>>> new
>>> > versions. We think it is time for Kylin to totally upgrade to the new
>>> > version. So that we can focus more on Kylin itself, instead of
>>> environments.
>>> > >
>>> > > Here is my proposal:
>>> > > 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a
>>> close
>>> > version);
>>> > > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous
>>> releases;
>>> > > 3) We will re-evaluate the need for building binary packages for
>>> > Cloudera release. (we may raise another discuss)
>>> > >
>>> > > Please let us know your comments. And please also understand with the
>>> > limited resource we couldn't support multiple Hadoop versions...
>>> > >
>>> > > Thanks!
>>> > >
>>> > > Best regards,
>>> > >
>>> > > Shaofeng Shi 史少锋
>>> > > Apache Kylin PMC
>>> > > Email: shaofengshi@apache.org
>>> > >
>>> > > Apache Kylin FAQ:
>>> https://kylin.apache.org/docs/gettingstarted/faq.html
>>> > > Join Kylin user mail group: user-subscribe@kylin.apache.org
>>> > > Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>>> > >
>>> > >
>>> >
>>> >
>>>
>>
Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Posted by Luke Han <lu...@gmail.com>.
+1 the new Kylin version should target Hadoop 3
Best Regards!
---------------------
Luke Han
On Mon, Mar 2, 2020 at 4:41 PM Billy Liu <bi...@apache.org> wrote:
> +1. Let's move to Hadoop3
>
> With Warm regards
>
> Billy Liu
>
>
> ShaoFeng Shi <sh...@apache.org> 于2020年2月27日周四 下午10:07写道：
>
>> Hi Yang,
>>
>> The main difference between 2.6 and 3.0 is the new real-time OLAP
>> feature. Hadoop 2 users can select either of them, depends on whether they
>> need the real-time feature.
>>
>> After 3.0, the next major features would be the Flink cube engine
>> (planned in v3.1) and the Parquet storage (early stage, maybe in v4.0).
>>
>> When the parquet storage is released, as the dependency on HBase can be
>> dropped, then we assume the API issue will easier than today. We can
>> re-evaluate the possibility to support Hadoop 2.
>>
>> So I think the impact on today's Hadoop 2 users is acceptable. Not
>> mention that they still can manually compile that.
>>
>>
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>> Apache Kylin PMC
>> Email: shaofengshi@apache.org
>>
>> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
>> Join Kylin user mail group: user-subscribe@kylin.apache.org
>> Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>>
>>
>>
>>
>> Li Yang <li...@apache.org> 于2020年2月27日周四 上午7:37写道：
>>
>>> The proposal means Kylin 3.0 will be the last major version that supports
>>> Hadoop 2.
>>>
>>> What will be recommended version for Hadoop 2 users after this? I feel
>>> the
>>> latest stable version of 2.6 is better than 3.0.
>>>
>>> Anyway, I'm fine with moving focus to Hadoop 3. That is the direction.
>>> However we shall also think about what it means for Hadoop 2 users.
>>> Questions like below shall also be answered.
>>>
>>> - What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not
>>> sound right here.)
>>> - How that version/branch will be maintained?
>>>
>>> +1 in general
>>>
>>> Regards
>>> -Yang
>>>
>>>
>>> On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang <zh...@outlook.com>
>>> wrote:
>>>
>>> > +1
>>> >
>>> >
>>> > > 2020年2月26日 下午3:48，ShaoFeng Shi <sh...@apache.org> 写道：
>>> > >
>>> > > Hello, Kylin users and developers,
>>> > >
>>> > > As we know Hadoop 3 and HBase 2 have released for some time. Kylin
>>> > starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of
>>> HBase
>>> > 1 and 2 are incompatible, we need to keep different branches for them.
>>> And
>>> > in each release, we need to build separate packages and do a round of
>>> > testing for them separately. Furthermore, Cloudera's API difference
>>> with
>>> > the Apache release makes the situation worse; We need to build 4 binary
>>> > packages for reach release. That has spent much of our manual effort
>>> and
>>> > computing resources.
>>> > >
>>> > > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for
>>> > production use; And we see more and more users are starting to use the
>>> new
>>> > versions. We think it is time for Kylin to totally upgrade to the new
>>> > version. So that we can focus more on Kylin itself, instead of
>>> environments.
>>> > >
>>> > > Here is my proposal:
>>> > > 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a
>>> close
>>> > version);
>>> > > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous
>>> releases;
>>> > > 3) We will re-evaluate the need for building binary packages for
>>> > Cloudera release. (we may raise another discuss)
>>> > >
>>> > > Please let us know your comments. And please also understand with the
>>> > limited resource we couldn't support multiple Hadoop versions...
>>> > >
>>> > > Thanks!
>>> > >
>>> > > Best regards,
>>> > >
>>> > > Shaofeng Shi 史少锋
>>> > > Apache Kylin PMC
>>> > > Email: shaofengshi@apache.org
>>> > >
>>> > > Apache Kylin FAQ:
>>> https://kylin.apache.org/docs/gettingstarted/faq.html
>>> > > Join Kylin user mail group: user-subscribe@kylin.apache.org
>>> > > Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>>> > >
>>> > >
>>> >
>>> >
>>>
>>
Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Posted by Luke Han <lu...@gmail.com>.
+1 the new Kylin version should target Hadoop 3
Best Regards!
---------------------
Luke Han
On Mon, Mar 2, 2020 at 4:41 PM Billy Liu <bi...@apache.org> wrote:
> +1. Let's move to Hadoop3
>
> With Warm regards
>
> Billy Liu
>
>
> ShaoFeng Shi <sh...@apache.org> 于2020年2月27日周四 下午10:07写道：
>
>> Hi Yang,
>>
>> The main difference between 2.6 and 3.0 is the new real-time OLAP
>> feature. Hadoop 2 users can select either of them, depends on whether they
>> need the real-time feature.
>>
>> After 3.0, the next major features would be the Flink cube engine
>> (planned in v3.1) and the Parquet storage (early stage, maybe in v4.0).
>>
>> When the parquet storage is released, as the dependency on HBase can be
>> dropped, then we assume the API issue will easier than today. We can
>> re-evaluate the possibility to support Hadoop 2.
>>
>> So I think the impact on today's Hadoop 2 users is acceptable. Not
>> mention that they still can manually compile that.
>>
>>
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>> Apache Kylin PMC
>> Email: shaofengshi@apache.org
>>
>> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
>> Join Kylin user mail group: user-subscribe@kylin.apache.org
>> Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>>
>>
>>
>>
>> Li Yang <li...@apache.org> 于2020年2月27日周四 上午7:37写道：
>>
>>> The proposal means Kylin 3.0 will be the last major version that supports
>>> Hadoop 2.
>>>
>>> What will be recommended version for Hadoop 2 users after this? I feel
>>> the
>>> latest stable version of 2.6 is better than 3.0.
>>>
>>> Anyway, I'm fine with moving focus to Hadoop 3. That is the direction.
>>> However we shall also think about what it means for Hadoop 2 users.
>>> Questions like below shall also be answered.
>>>
>>> - What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not
>>> sound right here.)
>>> - How that version/branch will be maintained?
>>>
>>> +1 in general
>>>
>>> Regards
>>> -Yang
>>>
>>>
>>> On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang <zh...@outlook.com>
>>> wrote:
>>>
>>> > +1
>>> >
>>> >
>>> > > 2020年2月26日 下午3:48，ShaoFeng Shi <sh...@apache.org> 写道：
>>> > >
>>> > > Hello, Kylin users and developers,
>>> > >
>>> > > As we know Hadoop 3 and HBase 2 have released for some time. Kylin
>>> > starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of
>>> HBase
>>> > 1 and 2 are incompatible, we need to keep different branches for them.
>>> And
>>> > in each release, we need to build separate packages and do a round of
>>> > testing for them separately. Furthermore, Cloudera's API difference
>>> with
>>> > the Apache release makes the situation worse; We need to build 4 binary
>>> > packages for reach release. That has spent much of our manual effort
>>> and
>>> > computing resources.
>>> > >
>>> > > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for
>>> > production use; And we see more and more users are starting to use the
>>> new
>>> > versions. We think it is time for Kylin to totally upgrade to the new
>>> > version. So that we can focus more on Kylin itself, instead of
>>> environments.
>>> > >
>>> > > Here is my proposal:
>>> > > 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a
>>> close
>>> > version);
>>> > > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous
>>> releases;
>>> > > 3) We will re-evaluate the need for building binary packages for
>>> > Cloudera release. (we may raise another discuss)
>>> > >
>>> > > Please let us know your comments. And please also understand with the
>>> > limited resource we couldn't support multiple Hadoop versions...
>>> > >
>>> > > Thanks!
>>> > >
>>> > > Best regards,
>>> > >
>>> > > Shaofeng Shi 史少锋
>>> > > Apache Kylin PMC
>>> > > Email: shaofengshi@apache.org
>>> > >
>>> > > Apache Kylin FAQ:
>>> https://kylin.apache.org/docs/gettingstarted/faq.html
>>> > > Join Kylin user mail group: user-subscribe@kylin.apache.org
>>> > > Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>>> > >
>>> > >
>>> >
>>> >
>>>
>>
Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Posted by Billy Liu <bi...@apache.org>.
+1. Let's move to Hadoop3
With Warm regards
Billy Liu
ShaoFeng Shi <sh...@apache.org> 于2020年2月27日周四 下午10:07写道：
> Hi Yang,
>
> The main difference between 2.6 and 3.0 is the new real-time OLAP feature.
> Hadoop 2 users can select either of them, depends on whether they need the
> real-time feature.
>
> After 3.0, the next major features would be the Flink cube engine (planned
> in v3.1) and the Parquet storage (early stage, maybe in v4.0).
>
> When the parquet storage is released, as the dependency on HBase can be
> dropped, then we assume the API issue will easier than today. We can
> re-evaluate the possibility to support Hadoop 2.
>
> So I think the impact on today's Hadoop 2 users is acceptable. Not mention
> that they still can manually compile that.
>
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofengshi@apache.org
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscribe@kylin.apache.org
> Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>
>
>
>
> Li Yang <li...@apache.org> 于2020年2月27日周四 上午7:37写道：
>
>> The proposal means Kylin 3.0 will be the last major version that supports
>> Hadoop 2.
>>
>> What will be recommended version for Hadoop 2 users after this? I feel the
>> latest stable version of 2.6 is better than 3.0.
>>
>> Anyway, I'm fine with moving focus to Hadoop 3. That is the direction.
>> However we shall also think about what it means for Hadoop 2 users.
>> Questions like below shall also be answered.
>>
>> - What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not
>> sound right here.)
>> - How that version/branch will be maintained?
>>
>> +1 in general
>>
>> Regards
>> -Yang
>>
>>
>> On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang <zh...@outlook.com> wrote:
>>
>> > +1
>> >
>> >
>> > > 2020年2月26日 下午3:48，ShaoFeng Shi <sh...@apache.org> 写道：
>> > >
>> > > Hello, Kylin users and developers,
>> > >
>> > > As we know Hadoop 3 and HBase 2 have released for some time. Kylin
>> > starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of
>> HBase
>> > 1 and 2 are incompatible, we need to keep different branches for them.
>> And
>> > in each release, we need to build separate packages and do a round of
>> > testing for them separately. Furthermore, Cloudera's API difference with
>> > the Apache release makes the situation worse; We need to build 4 binary
>> > packages for reach release. That has spent much of our manual effort and
>> > computing resources.
>> > >
>> > > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for
>> > production use; And we see more and more users are starting to use the
>> new
>> > versions. We think it is time for Kylin to totally upgrade to the new
>> > version. So that we can focus more on Kylin itself, instead of
>> environments.
>> > >
>> > > Here is my proposal:
>> > > 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a
>> close
>> > version);
>> > > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous releases;
>> > > 3) We will re-evaluate the need for building binary packages for
>> > Cloudera release. (we may raise another discuss)
>> > >
>> > > Please let us know your comments. And please also understand with the
>> > limited resource we couldn't support multiple Hadoop versions...
>> > >
>> > > Thanks!
>> > >
>> > > Best regards,
>> > >
>> > > Shaofeng Shi 史少锋
>> > > Apache Kylin PMC
>> > > Email: shaofengshi@apache.org
>> > >
>> > > Apache Kylin FAQ:
>> https://kylin.apache.org/docs/gettingstarted/faq.html
>> > > Join Kylin user mail group: user-subscribe@kylin.apache.org
>> > > Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>> > >
>> > >
>> >
>> >
>>
>
Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Posted by Billy Liu <bi...@apache.org>.
+1. Let's move to Hadoop3
With Warm regards
Billy Liu
ShaoFeng Shi <sh...@apache.org> 于2020年2月27日周四 下午10:07写道：
> Hi Yang,
>
> The main difference between 2.6 and 3.0 is the new real-time OLAP feature.
> Hadoop 2 users can select either of them, depends on whether they need the
> real-time feature.
>
> After 3.0, the next major features would be the Flink cube engine (planned
> in v3.1) and the Parquet storage (early stage, maybe in v4.0).
>
> When the parquet storage is released, as the dependency on HBase can be
> dropped, then we assume the API issue will easier than today. We can
> re-evaluate the possibility to support Hadoop 2.
>
> So I think the impact on today's Hadoop 2 users is acceptable. Not mention
> that they still can manually compile that.
>
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofengshi@apache.org
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscribe@kylin.apache.org
> Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>
>
>
>
> Li Yang <li...@apache.org> 于2020年2月27日周四 上午7:37写道：
>
>> The proposal means Kylin 3.0 will be the last major version that supports
>> Hadoop 2.
>>
>> What will be recommended version for Hadoop 2 users after this? I feel the
>> latest stable version of 2.6 is better than 3.0.
>>
>> Anyway, I'm fine with moving focus to Hadoop 3. That is the direction.
>> However we shall also think about what it means for Hadoop 2 users.
>> Questions like below shall also be answered.
>>
>> - What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not
>> sound right here.)
>> - How that version/branch will be maintained?
>>
>> +1 in general
>>
>> Regards
>> -Yang
>>
>>
>> On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang <zh...@outlook.com> wrote:
>>
>> > +1
>> >
>> >
>> > > 2020年2月26日 下午3:48，ShaoFeng Shi <sh...@apache.org> 写道：
>> > >
>> > > Hello, Kylin users and developers,
>> > >
>> > > As we know Hadoop 3 and HBase 2 have released for some time. Kylin
>> > starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of
>> HBase
>> > 1 and 2 are incompatible, we need to keep different branches for them.
>> And
>> > in each release, we need to build separate packages and do a round of
>> > testing for them separately. Furthermore, Cloudera's API difference with
>> > the Apache release makes the situation worse; We need to build 4 binary
>> > packages for reach release. That has spent much of our manual effort and
>> > computing resources.
>> > >
>> > > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for
>> > production use; And we see more and more users are starting to use the
>> new
>> > versions. We think it is time for Kylin to totally upgrade to the new
>> > version. So that we can focus more on Kylin itself, instead of
>> environments.
>> > >
>> > > Here is my proposal:
>> > > 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a
>> close
>> > version);
>> > > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous releases;
>> > > 3) We will re-evaluate the need for building binary packages for
>> > Cloudera release. (we may raise another discuss)
>> > >
>> > > Please let us know your comments. And please also understand with the
>> > limited resource we couldn't support multiple Hadoop versions...
>> > >
>> > > Thanks!
>> > >
>> > > Best regards,
>> > >
>> > > Shaofeng Shi 史少锋
>> > > Apache Kylin PMC
>> > > Email: shaofengshi@apache.org
>> > >
>> > > Apache Kylin FAQ:
>> https://kylin.apache.org/docs/gettingstarted/faq.html
>> > > Join Kylin user mail group: user-subscribe@kylin.apache.org
>> > > Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>> > >
>> > >
>> >
>> >
>>
>
Re: docker run error
Posted by ShaoFeng Shi <sh...@apache.org>.
I also tested it these days, it works; Anyway, the new 3.0.1 is also
updated. Please check it here:
https://github.com/apache/kylin/tree/master/docker
Best regards,
Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofengshi@apache.org
Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscribe@kylin.apache.org
Join Kylin dev mail group: dev-subscribe@kylin.apache.org
猫猫 <16...@qq.com> 于2020年2月28日周五 上午8:18写道：
> http://kylin.apache.org/docs/install/kylin_docker.html
>
> I use this docker image,but zookeeper don't run.
>
> docker pull apachekylin/apache-kylin-standalone:3.0.0-alpha2
>
> in entrypoint.sh file hasn't start zookeeper cmd.
>
> is this image disabled？
>
docker run error
Posted by 猫猫 <16...@qq.com>.
http://kylin.apache.org/docs/install/kylin_docker.html
I use this docker image,but zookeeper don't run.
docker pull apachekylin/apache-kylin-standalone:3.0.0-alpha2
in entrypoint.sh file hasn't start zookeeper cmd.
is this image disabled？
Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Yang,
The main difference between 2.6 and 3.0 is the new real-time OLAP feature.
Hadoop 2 users can select either of them, depends on whether they need the
real-time feature.
After 3.0, the next major features would be the Flink cube engine (planned
in v3.1) and the Parquet storage (early stage, maybe in v4.0).
When the parquet storage is released, as the dependency on HBase can be
dropped, then we assume the API issue will easier than today. We can
re-evaluate the possibility to support Hadoop 2.
So I think the impact on today's Hadoop 2 users is acceptable. Not mention
that they still can manually compile that.
Best regards,
Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofengshi@apache.org
Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscribe@kylin.apache.org
Join Kylin dev mail group: dev-subscribe@kylin.apache.org
Li Yang <li...@apache.org> 于2020年2月27日周四 上午7:37写道：
> The proposal means Kylin 3.0 will be the last major version that supports
> Hadoop 2.
>
> What will be recommended version for Hadoop 2 users after this? I feel the
> latest stable version of 2.6 is better than 3.0.
>
> Anyway, I'm fine with moving focus to Hadoop 3. That is the direction.
> However we shall also think about what it means for Hadoop 2 users.
> Questions like below shall also be answered.
>
> - What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not
> sound right here.)
> - How that version/branch will be maintained?
>
> +1 in general
>
> Regards
> -Yang
>
>
> On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang <zh...@outlook.com> wrote:
>
> > +1
> >
> >
> > > 2020年2月26日 下午3:48，ShaoFeng Shi <sh...@apache.org> 写道：
> > >
> > > Hello, Kylin users and developers,
> > >
> > > As we know Hadoop 3 and HBase 2 have released for some time. Kylin
> > starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of
> HBase
> > 1 and 2 are incompatible, we need to keep different branches for them.
> And
> > in each release, we need to build separate packages and do a round of
> > testing for them separately. Furthermore, Cloudera's API difference with
> > the Apache release makes the situation worse; We need to build 4 binary
> > packages for reach release. That has spent much of our manual effort and
> > computing resources.
> > >
> > > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for
> > production use; And we see more and more users are starting to use the
> new
> > versions. We think it is time for Kylin to totally upgrade to the new
> > version. So that we can focus more on Kylin itself, instead of
> environments.
> > >
> > > Here is my proposal:
> > > 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a
> close
> > version);
> > > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous releases;
> > > 3) We will re-evaluate the need for building binary packages for
> > Cloudera release. (we may raise another discuss)
> > >
> > > Please let us know your comments. And please also understand with the
> > limited resource we couldn't support multiple Hadoop versions...
> > >
> > > Thanks!
> > >
> > > Best regards,
> > >
> > > Shaofeng Shi 史少锋
> > > Apache Kylin PMC
> > > Email: shaofengshi@apache.org
> > >
> > > Apache Kylin FAQ:
> https://kylin.apache.org/docs/gettingstarted/faq.html
> > > Join Kylin user mail group: user-subscribe@kylin.apache.org
> > > Join Kylin dev mail group: dev-subscribe@kylin.apache.org
> > >
> > >
> >
> >
>
Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Yang,
The main difference between 2.6 and 3.0 is the new real-time OLAP feature.
Hadoop 2 users can select either of them, depends on whether they need the
real-time feature.
After 3.0, the next major features would be the Flink cube engine (planned
in v3.1) and the Parquet storage (early stage, maybe in v4.0).
When the parquet storage is released, as the dependency on HBase can be
dropped, then we assume the API issue will easier than today. We can
re-evaluate the possibility to support Hadoop 2.
So I think the impact on today's Hadoop 2 users is acceptable. Not mention
that they still can manually compile that.
Best regards,
Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofengshi@apache.org
Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscribe@kylin.apache.org
Join Kylin dev mail group: dev-subscribe@kylin.apache.org
Li Yang <li...@apache.org> 于2020年2月27日周四 上午7:37写道：
> The proposal means Kylin 3.0 will be the last major version that supports
> Hadoop 2.
>
> What will be recommended version for Hadoop 2 users after this? I feel the
> latest stable version of 2.6 is better than 3.0.
>
> Anyway, I'm fine with moving focus to Hadoop 3. That is the direction.
> However we shall also think about what it means for Hadoop 2 users.
> Questions like below shall also be answered.
>
> - What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not
> sound right here.)
> - How that version/branch will be maintained?
>
> +1 in general
>
> Regards
> -Yang
>
>
> On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang <zh...@outlook.com> wrote:
>
> > +1
> >
> >
> > > 2020年2月26日 下午3:48，ShaoFeng Shi <sh...@apache.org> 写道：
> > >
> > > Hello, Kylin users and developers,
> > >
> > > As we know Hadoop 3 and HBase 2 have released for some time. Kylin
> > starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of
> HBase
> > 1 and 2 are incompatible, we need to keep different branches for them.
> And
> > in each release, we need to build separate packages and do a round of
> > testing for them separately. Furthermore, Cloudera's API difference with
> > the Apache release makes the situation worse; We need to build 4 binary
> > packages for reach release. That has spent much of our manual effort and
> > computing resources.
> > >
> > > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for
> > production use; And we see more and more users are starting to use the
> new
> > versions. We think it is time for Kylin to totally upgrade to the new
> > version. So that we can focus more on Kylin itself, instead of
> environments.
> > >
> > > Here is my proposal:
> > > 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a
> close
> > version);
> > > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous releases;
> > > 3) We will re-evaluate the need for building binary packages for
> > Cloudera release. (we may raise another discuss)
> > >
> > > Please let us know your comments. And please also understand with the
> > limited resource we couldn't support multiple Hadoop versions...
> > >
> > > Thanks!
> > >
> > > Best regards,
> > >
> > > Shaofeng Shi 史少锋
> > > Apache Kylin PMC
> > > Email: shaofengshi@apache.org
> > >
> > > Apache Kylin FAQ:
> https://kylin.apache.org/docs/gettingstarted/faq.html
> > > Join Kylin user mail group: user-subscribe@kylin.apache.org
> > > Join Kylin dev mail group: dev-subscribe@kylin.apache.org
> > >
> > >
> >
> >
>
Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Posted by Li Yang <li...@apache.org>.
The proposal means Kylin 3.0 will be the last major version that supports
Hadoop 2.
What will be recommended version for Hadoop 2 users after this? I feel the
latest stable version of 2.6 is better than 3.0.
Anyway, I'm fine with moving focus to Hadoop 3. That is the direction.
However we shall also think about what it means for Hadoop 2 users.
Questions like below shall also be answered.
- What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not
sound right here.)
- How that version/branch will be maintained?
+1 in general
Regards
-Yang
On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang <zh...@outlook.com> wrote:
> +1
>
>
> > 2020年2月26日 下午3:48，ShaoFeng Shi <sh...@apache.org> 写道：
> >
> > Hello, Kylin users and developers,
> >
> > As we know Hadoop 3 and HBase 2 have released for some time. Kylin
> starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of HBase
> 1 and 2 are incompatible, we need to keep different branches for them. And
> in each release, we need to build separate packages and do a round of
> testing for them separately. Furthermore, Cloudera's API difference with
> the Apache release makes the situation worse; We need to build 4 binary
> packages for reach release. That has spent much of our manual effort and
> computing resources.
> >
> > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for
> production use; And we see more and more users are starting to use the new
> versions. We think it is time for Kylin to totally upgrade to the new
> version. So that we can focus more on Kylin itself, instead of environments.
> >
> > Here is my proposal:
> > 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a close
> version);
> > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous releases;
> > 3) We will re-evaluate the need for building binary packages for
> Cloudera release. (we may raise another discuss)
> >
> > Please let us know your comments. And please also understand with the
> limited resource we couldn't support multiple Hadoop versions...
> >
> > Thanks!
> >
> > Best regards,
> >
> > Shaofeng Shi 史少锋
> > Apache Kylin PMC
> > Email: shaofengshi@apache.org
> >
> > Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> > Join Kylin user mail group: user-subscribe@kylin.apache.org
> > Join Kylin dev mail group: dev-subscribe@kylin.apache.org
> >
> >
>
>
Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Posted by Li Yang <li...@apache.org>.
The proposal means Kylin 3.0 will be the last major version that supports
Hadoop 2.
What will be recommended version for Hadoop 2 users after this? I feel the
latest stable version of 2.6 is better than 3.0.
Anyway, I'm fine with moving focus to Hadoop 3. That is the direction.
However we shall also think about what it means for Hadoop 2 users.
Questions like below shall also be answered.
- What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not
sound right here.)
- How that version/branch will be maintained?
+1 in general
Regards
-Yang
On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang <zh...@outlook.com> wrote:
> +1
>
>
> > 2020年2月26日 下午3:48，ShaoFeng Shi <sh...@apache.org> 写道：
> >
> > Hello, Kylin users and developers,
> >
> > As we know Hadoop 3 and HBase 2 have released for some time. Kylin
> starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of HBase
> 1 and 2 are incompatible, we need to keep different branches for them. And
> in each release, we need to build separate packages and do a round of
> testing for them separately. Furthermore, Cloudera's API difference with
> the Apache release makes the situation worse; We need to build 4 binary
> packages for reach release. That has spent much of our manual effort and
> computing resources.
> >
> > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for
> production use; And we see more and more users are starting to use the new
> versions. We think it is time for Kylin to totally upgrade to the new
> version. So that we can focus more on Kylin itself, instead of environments.
> >
> > Here is my proposal:
> > 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a close
> version);
> > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous releases;
> > 3) We will re-evaluate the need for building binary packages for
> Cloudera release. (we may raise another discuss)
> >
> > Please let us know your comments. And please also understand with the
> limited resource we couldn't support multiple Hadoop versions...
> >
> > Thanks!
> >
> > Best regards,
> >
> > Shaofeng Shi 史少锋
> > Apache Kylin PMC
> > Email: shaofengshi@apache.org
> >
> > Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> > Join Kylin user mail group: user-subscribe@kylin.apache.org
> > Join Kylin dev mail group: dev-subscribe@kylin.apache.org
> >
> >
>
>
Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Posted by Zhou Kang <zh...@outlook.com>.
+1
> 2020年2月26日 下午3:48，ShaoFeng Shi <sh...@apache.org> 写道：
>
> Hello, Kylin users and developers,
>
> As we know Hadoop 3 and HBase 2 have released for some time. Kylin starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of HBase 1 and 2 are incompatible, we need to keep different branches for them. And in each release, we need to build separate packages and do a round of testing for them separately. Furthermore, Cloudera's API difference with the Apache release makes the situation worse; We need to build 4 binary packages for reach release. That has spent much of our manual effort and computing resources.
>
> Today, Hadoop 3 + HBase 2 becomes enough mature and stable for production use; And we see more and more users are starting to use the new versions. We think it is time for Kylin to totally upgrade to the new version. So that we can focus more on Kylin itself, instead of environments.
>
> Here is my proposal:
> 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a close version);
> 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous releases;
> 3) We will re-evaluate the need for building binary packages for Cloudera release. (we may raise another discuss)
>
> Please let us know your comments. And please also understand with the limited resource we couldn't support multiple Hadoop versions...
>
> Thanks!
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofengshi@apache.org
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscribe@kylin.apache.org
> Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>
>
Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Posted by Zhou Kang <zh...@outlook.com>.
+1
> 2020年2月26日 下午3:48，ShaoFeng Shi <sh...@apache.org> 写道：
>
> Hello, Kylin users and developers,
>
> As we know Hadoop 3 and HBase 2 have released for some time. Kylin starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of HBase 1 and 2 are incompatible, we need to keep different branches for them. And in each release, we need to build separate packages and do a round of testing for them separately. Furthermore, Cloudera's API difference with the Apache release makes the situation worse; We need to build 4 binary packages for reach release. That has spent much of our manual effort and computing resources.
>
> Today, Hadoop 3 + HBase 2 becomes enough mature and stable for production use; And we see more and more users are starting to use the new versions. We think it is time for Kylin to totally upgrade to the new version. So that we can focus more on Kylin itself, instead of environments.
>
> Here is my proposal:
> 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a close version);
> 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous releases;
> 3) We will re-evaluate the need for building binary packages for Cloudera release. (we may raise another discuss)
>
> Please let us know your comments. And please also understand with the limited resource we couldn't support multiple Hadoop versions...
>
> Thanks!
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofengshi@apache.org
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscribe@kylin.apache.org
> Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>
>