You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@doris.apache.org by GuoLei Yi <yi...@gmail.com> on 2022/06/25 12:00:59 UTC
Remove alpha rowset or segment v1 related code.
Hi guys,
I want to remove AlphaRowset or Segment V1 related code because :
1. Many optimizations like vectorization or low cardinality.
2. New types is not implemented in segment v1 such as array type, json
type, date v2 or decimal v3.
3. We need modify alpha rowset related code when we modify vectorization
code like predicate. It is very hard to maintain these code.
We have already add checks and auto conversions in doris 1.1. Could refer
this issue https://github.com/apache/doris/issues/8984.
Any advice is appreciated.
--
衣国垒
SelectDB
Tel:134-3991-0228
Email:yiguolei@gmail.com
Re: Remove alpha rowset or segment v1 related code.
Posted by Gabriel Lee <ga...@gmail.com>.
Big step! Look forward to it!
Best,
Gabriel
On Sat, 25 Jun 2022 at 20:01, GuoLei Yi <yi...@gmail.com> wrote:
> Hi guys,
>
> I want to remove AlphaRowset or Segment V1 related code because :
> 1. Many optimizations like vectorization or low cardinality.
> 2. New types is not implemented in segment v1 such as array type, json
> type, date v2 or decimal v3.
> 3. We need modify alpha rowset related code when we modify vectorization
> code like predicate. It is very hard to maintain these code.
>
> We have already add checks and auto conversions in doris 1.1. Could refer
> this issue https://github.com/apache/doris/issues/8984.
>
> Any advice is appreciated.
>
> --
> 衣国垒
> SelectDB
> Tel:134-3991-0228
> Email:yiguolei@gmail.com
>
Re: Re: Remove alpha rowset or segment v1 related code.
Posted by zhg yang <ya...@gmail.com>.
It is now possible to detect the format of the tablet in the database at
that time through ADMIN SHOW TABLET STORAGE FORMAT, but there is no tools
to explicitly convert v1 to v2, we need to provide such a tool before
removing the code
Thanks
Yang Zhengguo
陈明雨 <mo...@163.com> 于2022年6月26日周日 11:12写道：
> Hi Guolei:
> Please write a detailed document to help users with the transition.
> including but not limited to:
>
>
> 1. How to confirm that there is still v1 version data in the cluster.
> 2. If so, how can data from v1 be converted to v2?
> 3. The possible impact on the business during the conversion process, such
> as disk space, resource overhead, etc.
> 4. Which version will not support reading and writing v1 data at all, and
> the detail upgrade steps
>
>
>
>
> --
>
> 此致！Best Regards
> 陈明雨 Mingyu Chen
>
> Email:
> morningman@apache.org
>
>
>
>
>
> 在 2022-06-25 21:00:27，"张家峰" <zh...@gmail.com> 写道：
> >+1
> >
> >Since this code is relatively early, and everyone is using the v2 version
> >now, it can be removed. In order to prepare users for the v1->v2
> >conversion, it can be explained in advance when this version is released.
> >We will Remove this part of the code in the next version, so everyone is
> >ready.
> >
> >jiafeng.Zhang / 张家锋
> >Email：jiafengzheng@apache.org
> >
> >
> >GuoLei Yi <yi...@gmail.com>于2022年6月25日 周六20:01写道：
> >
> >> Hi guys,
> >>
> >> I want to remove AlphaRowset or Segment V1 related code because :
> >> 1. Many optimizations like vectorization or low cardinality.
> >> 2. New types is not implemented in segment v1 such as array type, json
> >> type, date v2 or decimal v3.
> >> 3. We need modify alpha rowset related code when we modify vectorization
> >> code like predicate. It is very hard to maintain these code.
> >>
> >> We have already add checks and auto conversions in doris 1.1. Could
> refer
> >> this issue https://github.com/apache/doris/issues/8984.
> >>
> >> Any advice is appreciated.
> >>
> >> --
> >> 衣国垒
> >> SelectDB
> >> Tel:134-3991-0228
> >> Email:yiguolei@gmail.com
> >>
> >--
> >张家峰
>
Re:Re: Remove alpha rowset or segment v1 related code.
Posted by 陈明雨 <mo...@163.com>.
Hi Guolei:
Please write a detailed document to help users with the transition. including but not limited to:
1. How to confirm that there is still v1 version data in the cluster.
2. If so, how can data from v1 be converted to v2?
3. The possible impact on the business during the conversion process, such as disk space, resource overhead, etc.
4. Which version will not support reading and writing v1 data at all, and the detail upgrade steps
--
此致！Best Regards
陈明雨 Mingyu Chen
Email:
morningman@apache.org
在 2022-06-25 21:00:27，"张家峰" <zh...@gmail.com> 写道：
>+1
>
>Since this code is relatively early, and everyone is using the v2 version
>now, it can be removed. In order to prepare users for the v1->v2
>conversion, it can be explained in advance when this version is released.
>We will Remove this part of the code in the next version, so everyone is
>ready.
>
>jiafeng.Zhang / 张家锋
>Email：jiafengzheng@apache.org
>
>
>GuoLei Yi <yi...@gmail.com>于2022年6月25日 周六20:01写道：
>
>> Hi guys,
>>
>> I want to remove AlphaRowset or Segment V1 related code because :
>> 1. Many optimizations like vectorization or low cardinality.
>> 2. New types is not implemented in segment v1 such as array type, json
>> type, date v2 or decimal v3.
>> 3. We need modify alpha rowset related code when we modify vectorization
>> code like predicate. It is very hard to maintain these code.
>>
>> We have already add checks and auto conversions in doris 1.1. Could refer
>> this issue https://github.com/apache/doris/issues/8984.
>>
>> Any advice is appreciated.
>>
>> --
>> 衣国垒
>> SelectDB
>> Tel:134-3991-0228
>> Email:yiguolei@gmail.com
>>
>--
>张家峰
Re: Remove alpha rowset or segment v1 related code.
Posted by 张家峰 <zh...@gmail.com>.
+1
Since this code is relatively early, and everyone is using the v2 version
now, it can be removed. In order to prepare users for the v1->v2
conversion, it can be explained in advance when this version is released.
We will Remove this part of the code in the next version, so everyone is
ready.
jiafeng.Zhang / 张家锋
Email：jiafengzheng@apache.org
GuoLei Yi <yi...@gmail.com>于2022年6月25日 周六20:01写道：
> Hi guys,
>
> I want to remove AlphaRowset or Segment V1 related code because :
> 1. Many optimizations like vectorization or low cardinality.
> 2. New types is not implemented in segment v1 such as array type, json
> type, date v2 or decimal v3.
> 3. We need modify alpha rowset related code when we modify vectorization
> code like predicate. It is very hard to maintain these code.
>
> We have already add checks and auto conversions in doris 1.1. Could refer
> this issue https://github.com/apache/doris/issues/8984.
>
> Any advice is appreciated.
>
> --
> 衣国垒
> SelectDB
> Tel:134-3991-0228
> Email:yiguolei@gmail.com
>
--
张家峰