You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@doris.apache.org by 晖 <82...@qq.com.INVALID> on 2022/03/29 12:55:56 UTC
回复： [Discuss][DSIP] Support load binlog from MySQL directly without relying other service
Hi Ling,
Do you mean 'canal' component? I have not cancelled any component,
only extended current binlog load with a new type of 'debezium'.
Because the Canal service is not provided in my company's product environment,
users can't use the current binlog load. I think the new type simplifies the usage
of binlog load, and shortens the data flow.
The similiar function is provided by ClickHouse. I think it's a brilliant experience to
use a MPP database as a MySQL's slave. It's not the same background with Flink
Connector.
------------------ 原始邮件 ------------------
发件人: "dev" <lingmiao@apache.org>;
发送时间: 2022年3月29日(星期二) 晚上8:13
收件人: "dev"<dev@doris.apache.org>;
主题: Re: [Discuss][DSIP] Support load binlog from MySQL directly without relying other service
Hi hui,
I want to know what is the reason for canceling the cancel component? Is
there any problem in use? What are the benefits ?
Parsing binlog is not actually the function of Doris itself. Or is it
better to appear as a connector? For example, like Flink connector.
Ling Miao
陈明雨 <morningman@163.com> 于2022年3月29日周二 19:46写道：
> Hi Hui,
> Thanks for it. Could you register a cwiki acount and tell me your account
> id? So that I can create DSIP and grant write priv for you.
> https://cwiki.apache.org/confluence/signup.action
>
>
>
>
> --
>
> 此致！Best Regards
> 陈明雨 Mingyu Chen
>
> Email:
> chenmingyu@apache.org
>
>
>
>
>
> At 2022-03-29 18:17:02, "晖" <827677355@qq.com.INVALID> wrote:
> >As issue #8025(
> https://github.com/apache/incubator-doris/issues/8025)&nbsp;mentioned,
> the current binlog load is relying on Canal service. We can integrate the
> binlog aquire and parse function into Doris, so that user can be more
> convinience to use it.Please determine the final DSIP number for it.
>
Re:Re: [Discuss][DSIP] Support load binlog from MySQL directly without relying other service
Posted by 蔡聪辉 <ca...@163.com>.
I think one of the elegant implementations may be something like routine load
在 2022-03-30 10:47:39，"ling miao" <li...@apache.org> 写道：
>> Do you mean 'canal' component?
>Yes.
>
>It sounds like you mean a new way about binlog load?
>You can show the architecture diagram and general design on the wiki, so
>that I can better understand your intentions.[1]
>
>Ling Miao
>[
>https://cwiki.apache.org/confluence/display/DORIS/DSIP-003%3A+Directly+binlog+consumption
>]
>
>晖 <82...@qq.com.invalid> 于2022年3月29日周二 20:56写道：
>
>> Hi Ling,
>> Do you mean 'canal' component? I have not cancelled any component,
>> only extended current binlog load with a new type of 'debezium'.
>>
>>
>> Because the Canal service is not provided in my company's product
>> environment,
>> users can't use the current binlog load. I think the new type simplifies
>> the usage
>> of binlog load, and shortens the data flow.
>>
>>
>> The similiar function is provided by ClickHouse. I think it's a brilliant
>> experience to
>> use a MPP database as a MySQL's slave. It's not the same background with
>> Flink
>> Connector.
>>
>>
>>
>>
>> ------------------ 原始邮件 ------------------
>> 发件人:
>> "dev"
>> <
>> lingmiao@apache.org>;
>> 发送时间: 2022年3月29日(星期二) 晚上8:13
>> 收件人: "dev"<dev@doris.apache.org>;
>>
>> 主题: Re: [Discuss][DSIP] Support load binlog from MySQL directly
>> without relying other service
>>
>>
>>
>> Hi hui,
>>
>> I want to know what is the reason for canceling the cancel component? Is
>> there any problem in use? What are the benefits ?
>>
>> Parsing binlog is not actually the function of Doris itself. Or is it
>> better to appear as a connector? For example, like Flink connector.
>>
>> Ling Miao
>>
>> 陈明雨 <morningman@163.com> 于2022年3月29日周二 19:46写道：
>>
>> > Hi Hui,
>> > Thanks for it. Could you register a cwiki acount and tell me your
>> account
>> > id? So that I can create DSIP and grant write priv for you.
>> > https://cwiki.apache.org/confluence/signup.action
>> >
>> >
>> >
>> >
>> > --
>> >
>> > 此致！Best Regards
>> > 陈明雨 Mingyu Chen
>> >
>> > Email:
>> > chenmingyu@apache.org
>> >
>> >
>> >
>> >
>> >
>> > At 2022-03-29 18:17:02, "晖" <827677355@qq.com.INVALID> wrote:
>> > >As issue #8025(
>> >
>> https://github.com/apache/incubator-doris/issues/8025)&nbsp;mentioned,
>> > the current binlog load is relying on Canal service. We can integrate
>> the
>> > binlog aquire and parse function into Doris, so that user can be more
>> > convinience to use it.Please determine the final DSIP number for it.
>> >
Re: [Discuss][DSIP] Support load binlog from MySQL directly without relying other service
Posted by ling miao <li...@apache.org>.
> Do you mean 'canal' component?
Yes.
It sounds like you mean a new way about binlog load?
You can show the architecture diagram and general design on the wiki, so
that I can better understand your intentions.[1]
Ling Miao
[
https://cwiki.apache.org/confluence/display/DORIS/DSIP-003%3A+Directly+binlog+consumption
]
晖 <82...@qq.com.invalid> 于2022年3月29日周二 20:56写道：
> Hi Ling,
> Do you mean 'canal' component? I have not cancelled any component,
> only extended current binlog load with a new type of 'debezium'.
>
>
> Because the Canal service is not provided in my company's product
> environment,
> users can't use the current binlog load. I think the new type simplifies
> the usage
> of binlog load, and shortens the data flow.
>
>
> The similiar function is provided by ClickHouse. I think it's a brilliant
> experience to
> use a MPP database as a MySQL's slave. It's not the same background with
> Flink
> Connector.
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人:
> "dev"
> <
> lingmiao@apache.org>;
> 发送时间: 2022年3月29日(星期二) 晚上8:13
> 收件人: "dev"<dev@doris.apache.org>;
>
> 主题: Re: [Discuss][DSIP] Support load binlog from MySQL directly
> without relying other service
>
>
>
> Hi hui,
>
> I want to know what is the reason for canceling the cancel component? Is
> there any problem in use? What are the benefits ?
>
> Parsing binlog is not actually the function of Doris itself. Or is it
> better to appear as a connector? For example, like Flink connector.
>
> Ling Miao
>
> 陈明雨 <morningman@163.com> 于2022年3月29日周二 19:46写道：
>
> > Hi Hui,
> > Thanks for it. Could you register a cwiki acount and tell me your
> account
> > id? So that I can create DSIP and grant write priv for you.
> > https://cwiki.apache.org/confluence/signup.action
> >
> >
> >
> >
> > --
> >
> > 此致！Best Regards
> > 陈明雨 Mingyu Chen
> >
> > Email:
> > chenmingyu@apache.org
> >
> >
> >
> >
> >
> > At 2022-03-29 18:17:02, "晖" <827677355@qq.com.INVALID> wrote:
> > >As issue #8025(
> >
> https://github.com/apache/incubator-doris/issues/8025)&nbsp;mentioned,
> > the current binlog load is relying on Canal service. We can integrate
> the
> > binlog aquire and parse function into Doris, so that user can be more
> > convinience to use it.Please determine the final DSIP number for it.
> >