You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@griffin.apache.org by William Guo <gu...@apache.org> on 2022/08/26 03:46:04 UTC
[DISCUSS] Griffin new arch
hi all,
Some dev might also be interested to review our proposal for a new arch.
Your thoughts?
Please check this link to review.
https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
Thanks,
William
Re: [DISCUSS] Griffin new arch
Posted by William Guo <gu...@apache.org>.
hi Warden,
Thanks for initiating this.
Since you mentioned interaction between core and dispatcher, I will start a
new thread attached with your protocol.
Thanks,
William
On Wed, Oct 12, 2022 at 10:48 AM Wang Warden <wa...@gmail.com> wrote:
> Hi,
>
> I have drafted the communication messages between core and dispatcher as
> below,
>
>
> // submit sql
> execute(RecordRequest) return RecordResponse;
>
> RecordRequest {
> String recordSql;
> Enum engine; // Spark,Hive,Presto,etc.
> String owner;
> Integer maxRetryCount;
> }
>
> RecordResponse {
> Integer code;
> Long value;
> Enum errorCode; // if code != 200, please tell us what happened:
> // 1、 recordSql syntax error
> // 2、 internal error, dispatcher self is crashed
> // 3、 external error, target engine is crashed when dispatcher
> call,etc
> Exception ex; // error detail info
> }
>
> // validate sql syntax
> validateSQL(CheckRequest) return CheckResponse;
>
> ValidateSQLRequest {
> String recordSql;
> Enum engine; // Spark,Hive,Presto,etc.
> }
>
> ValidateSQLResponse {
> Integer code;
> Enum errorCode; // if code != 200, please tell us what happened:
> // 1、 recordSql syntax error
> // 2、 internal error, dispatcher self is crashed
> // 3、 external error, target engine is crashed when dispatcher
> call,etc
> Exception ex; // error detail info
> }
>
>
>
> Could you please review it? Give us your feedback.
>
> Thanks,
> Wang
>
> Willem Jiang <wi...@gmail.com> 於 2022年10月11日 週二 下午4:53寫道：
>
>> Great, thanks for the update.
>>
>> Willem Jiang
>>
>> Twitter: willemjiang
>> Weibo: 姜宁willem
>>
>> On Tue, Oct 11, 2022 at 1:13 PM William Guo <gu...@apache.org> wrote:
>> >
>> > hi Willem,
>> >
>> > I'm preparing the roadmap in our jira, will finish it tomorrow.
>> >
>> > Thanks,
>> > William
>> >
>> > On Tue, Oct 11, 2022 at 10:21 AM Willem Jiang <wi...@gmail.com>
>> wrote:
>> >>
>> >> It is good to see we are planning a new version of the griffin.
>> >> I think we can write a roadmap to help users and other contributors to
>> >> know better about the whole process.
>> >>
>> >> Willem Jiang
>> >>
>> >> Twitter: willemjiang
>> >> Weibo: 姜宁willem
>> >>
>> >> On Mon, Oct 10, 2022 at 11:52 PM ding wang <wa...@gmail.com>
>> wrote:
>> >> >
>> >> > hi jianhua
>> >> >
>> >> > I would like to join to implement the core module, and since it
>> needs to
>> >> > communicate with dispatcher module.
>> >> >
>> >> > I will provide the communication protocol(interface) between core and
>> >> > dispatcher according to the architecture.
>> >> >
>> >> > Thanks,
>> >> >
>> >> > Wang
>> >> >
>> >> > William Guo <gu...@apache.org> 於 2022年10月10日 週一 晚上7:42寫道：
>> >> >
>> >> > > Cool.
>> >> > >
>> >> > > Before we discuss, could you please document dispatcher design for
>> us to
>> >> > > review.
>> >> > >
>> >> > > Thanks,
>> >> > >
>> >> > >
>> >> > >
>> >> > > On Mon, Oct 10, 2022 at 6:20 PM jianhua guo <gu...@gmail.com>
>> wrote:
>> >> > >
>> >> > > > hi all,
>> >> > > >
>> >> > > > I have reviewed the new arch, and will try to implement the
>> dispatcher
>> >> > > > module in the coming days.
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
>> >> > > >
>> >> > > > https://github.com/apache/griffin/tree/griffin-1.0.0-dev
>> >> > > >
>> >> > > > I need to discuss with you the dispatcher interface with the
>> core module.
>> >> > > >
>> >> > > >
>> >> > > > Thanks,
>> >> > > >
>> >> > > > Jianhua
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > On Wed, Aug 31, 2022 at 6:54 AM William Guo <gu...@apache.org>
>> wrote:
>> >> > > >
>> >> > > > > hi Eugene,
>> >> > > > >
>> >> > > > > 1. We need to seamlessly migrate from old tasks to new
>> workflows, some
>> >> > > > > modules will be optional in new workflows since old tasks lack
>> them.
>> >> > > > > 2. I will draw some diagrams to illustrate it.
>> >> > > > >
>> >> > > > > Thanks,
>> >> > > > > William
>> >> > > > >
>> >> > > > > On Sun, Aug 28, 2022 at 5:07 PM Eugene Law <li...@apache.org>
>> wrote:
>> >> > > > >
>> >> > > > > > Hi William
>> >> > > > > >
>> >> > > > > > Raised a few comments on arch, replaying here
>> >> > > > > > 1.Do we need to consider upgrading from old arch to new
>> arch, like
>> >> > > how
>> >> > > > to
>> >> > > > > > seamlessly migrate old Griffin tasks into the new one?
>> >> > > > > > 2.The arch diagram is focusing on service relationships, can
>> you add
>> >> > > > > > component or deployment diagram to describe it from
>> different angles?
>> >> > > > > >
>> >> > > > > > Thx
>> >> > > > > >
>> >> > > > > > On Sun, Aug 28, 2022 at 2:31 PM William Guo <
>> guoyp@apache.org>
>> >> > > wrote:
>> >> > > > > >
>> >> > > > > > > Hi Eugene,
>> >> > > > > > >
>> >> > > > > > > Granted.
>> >> > > > > > > Everyone can comment on our space.
>> >> > > > > > >
>> >> > > > > > > Thanks,
>> >> > > > > > > William
>> >> > > > > > >
>> >> > > > > > > On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <
>> liujin@apache.org>
>> >> > > > wrote:
>> >> > > > > > >
>> >> > > > > > > > Hi William
>> >> > > > > > > >
>> >> > > > > > > > Could you grant comment permission on the confluence
>> page? so it
>> >> > > > > will
>> >> > > > > > > ease
>> >> > > > > > > > the conversations of proposal discussion among the
>> Griffin
>> >> > > > community.
>> >> > > > > > > >
>> >> > > > > > > > Thx
>> >> > > > > > > >
>> >> > > > > > > >
>> >> > > > > > > > On Fri, Aug 26, 2022 at 11:46 AM William Guo <
>> guoyp@apache.org>
>> >> > > > > wrote:
>> >> > > > > > > >
>> >> > > > > > > > > hi all,
>> >> > > > > > > > >
>> >> > > > > > > > > Some dev might also be interested to review our
>> proposal for a
>> >> > > > new
>> >> > > > > > > arch.
>> >> > > > > > > > > Your thoughts?
>> >> > > > > > > > >
>> >> > > > > > > > > Please check this link to review.
>> >> > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
>> >> > > > > > > > >
>> >> > > > > > > > > Thanks,
>> >> > > > > > > > > William
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>>
>
Re: [DISCUSS] Griffin new arch
Posted by Wang Warden <wa...@gmail.com>.
Hi,
I have drafted the communication messages between core and dispatcher as
below,
// submit sql
execute(RecordRequest) return RecordResponse;
RecordRequest {
String recordSql;
Enum engine; // Spark,Hive,Presto,etc.
String owner;
Integer maxRetryCount;
}
RecordResponse {
Integer code;
Long value;
Enum errorCode; // if code != 200, please tell us what happened:
// 1、 recordSql syntax error
// 2、 internal error, dispatcher self is crashed
// 3、 external error, target engine is crashed when dispatcher
call,etc
Exception ex; // error detail info
}
// validate sql syntax
validateSQL(CheckRequest) return CheckResponse;
ValidateSQLRequest {
String recordSql;
Enum engine; // Spark,Hive,Presto,etc.
}
ValidateSQLResponse {
Integer code;
Enum errorCode; // if code != 200, please tell us what happened:
// 1、 recordSql syntax error
// 2、 internal error, dispatcher self is crashed
// 3、 external error, target engine is crashed when dispatcher
call,etc
Exception ex; // error detail info
}
Could you please review it? Give us your feedback.
Thanks,
Wang
Willem Jiang <wi...@gmail.com> 於 2022年10月11日 週二 下午4:53寫道：
> Great, thanks for the update.
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Tue, Oct 11, 2022 at 1:13 PM William Guo <gu...@apache.org> wrote:
> >
> > hi Willem,
> >
> > I'm preparing the roadmap in our jira, will finish it tomorrow.
> >
> > Thanks,
> > William
> >
> > On Tue, Oct 11, 2022 at 10:21 AM Willem Jiang <wi...@gmail.com>
> wrote:
> >>
> >> It is good to see we are planning a new version of the griffin.
> >> I think we can write a roadmap to help users and other contributors to
> >> know better about the whole process.
> >>
> >> Willem Jiang
> >>
> >> Twitter: willemjiang
> >> Weibo: 姜宁willem
> >>
> >> On Mon, Oct 10, 2022 at 11:52 PM ding wang <wa...@gmail.com>
> wrote:
> >> >
> >> > hi jianhua
> >> >
> >> > I would like to join to implement the core module, and since it needs
> to
> >> > communicate with dispatcher module.
> >> >
> >> > I will provide the communication protocol(interface) between core and
> >> > dispatcher according to the architecture.
> >> >
> >> > Thanks,
> >> >
> >> > Wang
> >> >
> >> > William Guo <gu...@apache.org> 於 2022年10月10日 週一 晚上7:42寫道：
> >> >
> >> > > Cool.
> >> > >
> >> > > Before we discuss, could you please document dispatcher design for
> us to
> >> > > review.
> >> > >
> >> > > Thanks,
> >> > >
> >> > >
> >> > >
> >> > > On Mon, Oct 10, 2022 at 6:20 PM jianhua guo <gu...@gmail.com>
> wrote:
> >> > >
> >> > > > hi all,
> >> > > >
> >> > > > I have reviewed the new arch, and will try to implement the
> dispatcher
> >> > > > module in the coming days.
> >> > > >
> >> > > >
> >> > > >
> >> > >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> >> > > >
> >> > > > https://github.com/apache/griffin/tree/griffin-1.0.0-dev
> >> > > >
> >> > > > I need to discuss with you the dispatcher interface with the core
> module.
> >> > > >
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > Jianhua
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Wed, Aug 31, 2022 at 6:54 AM William Guo <gu...@apache.org>
> wrote:
> >> > > >
> >> > > > > hi Eugene,
> >> > > > >
> >> > > > > 1. We need to seamlessly migrate from old tasks to new
> workflows, some
> >> > > > > modules will be optional in new workflows since old tasks lack
> them.
> >> > > > > 2. I will draw some diagrams to illustrate it.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > William
> >> > > > >
> >> > > > > On Sun, Aug 28, 2022 at 5:07 PM Eugene Law <li...@apache.org>
> wrote:
> >> > > > >
> >> > > > > > Hi William
> >> > > > > >
> >> > > > > > Raised a few comments on arch, replaying here
> >> > > > > > 1.Do we need to consider upgrading from old arch to new arch,
> like
> >> > > how
> >> > > > to
> >> > > > > > seamlessly migrate old Griffin tasks into the new one?
> >> > > > > > 2.The arch diagram is focusing on service relationships, can
> you add
> >> > > > > > component or deployment diagram to describe it from different
> angles?
> >> > > > > >
> >> > > > > > Thx
> >> > > > > >
> >> > > > > > On Sun, Aug 28, 2022 at 2:31 PM William Guo <guoyp@apache.org
> >
> >> > > wrote:
> >> > > > > >
> >> > > > > > > Hi Eugene,
> >> > > > > > >
> >> > > > > > > Granted.
> >> > > > > > > Everyone can comment on our space.
> >> > > > > > >
> >> > > > > > > Thanks,
> >> > > > > > > William
> >> > > > > > >
> >> > > > > > > On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <
> liujin@apache.org>
> >> > > > wrote:
> >> > > > > > >
> >> > > > > > > > Hi William
> >> > > > > > > >
> >> > > > > > > > Could you grant comment permission on the confluence
> page? so it
> >> > > > > will
> >> > > > > > > ease
> >> > > > > > > > the conversations of proposal discussion among the Griffin
> >> > > > community.
> >> > > > > > > >
> >> > > > > > > > Thx
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > On Fri, Aug 26, 2022 at 11:46 AM William Guo <
> guoyp@apache.org>
> >> > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > hi all,
> >> > > > > > > > >
> >> > > > > > > > > Some dev might also be interested to review our
> proposal for a
> >> > > > new
> >> > > > > > > arch.
> >> > > > > > > > > Your thoughts?
> >> > > > > > > > >
> >> > > > > > > > > Please check this link to review.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> >> > > > > > > > >
> >> > > > > > > > > Thanks,
> >> > > > > > > > > William
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
>
Re: [DISCUSS] Griffin new arch
Posted by Willem Jiang <wi...@gmail.com>.
Great, thanks for the update.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Tue, Oct 11, 2022 at 1:13 PM William Guo <gu...@apache.org> wrote:
>
> hi Willem,
>
> I'm preparing the roadmap in our jira, will finish it tomorrow.
>
> Thanks,
> William
>
> On Tue, Oct 11, 2022 at 10:21 AM Willem Jiang <wi...@gmail.com> wrote:
>>
>> It is good to see we are planning a new version of the griffin.
>> I think we can write a roadmap to help users and other contributors to
>> know better about the whole process.
>>
>> Willem Jiang
>>
>> Twitter: willemjiang
>> Weibo: 姜宁willem
>>
>> On Mon, Oct 10, 2022 at 11:52 PM ding wang <wa...@gmail.com> wrote:
>> >
>> > hi jianhua
>> >
>> > I would like to join to implement the core module, and since it needs to
>> > communicate with dispatcher module.
>> >
>> > I will provide the communication protocol(interface) between core and
>> > dispatcher according to the architecture.
>> >
>> > Thanks,
>> >
>> > Wang
>> >
>> > William Guo <gu...@apache.org> 於 2022年10月10日 週一 晚上7:42寫道：
>> >
>> > > Cool.
>> > >
>> > > Before we discuss, could you please document dispatcher design for us to
>> > > review.
>> > >
>> > > Thanks,
>> > >
>> > >
>> > >
>> > > On Mon, Oct 10, 2022 at 6:20 PM jianhua guo <gu...@gmail.com> wrote:
>> > >
>> > > > hi all,
>> > > >
>> > > > I have reviewed the new arch, and will try to implement the dispatcher
>> > > > module in the coming days.
>> > > >
>> > > >
>> > > >
>> > > https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
>> > > >
>> > > > https://github.com/apache/griffin/tree/griffin-1.0.0-dev
>> > > >
>> > > > I need to discuss with you the dispatcher interface with the core module.
>> > > >
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Jianhua
>> > > >
>> > > >
>> > > >
>> > > > On Wed, Aug 31, 2022 at 6:54 AM William Guo <gu...@apache.org> wrote:
>> > > >
>> > > > > hi Eugene,
>> > > > >
>> > > > > 1. We need to seamlessly migrate from old tasks to new workflows, some
>> > > > > modules will be optional in new workflows since old tasks lack them.
>> > > > > 2. I will draw some diagrams to illustrate it.
>> > > > >
>> > > > > Thanks,
>> > > > > William
>> > > > >
>> > > > > On Sun, Aug 28, 2022 at 5:07 PM Eugene Law <li...@apache.org> wrote:
>> > > > >
>> > > > > > Hi William
>> > > > > >
>> > > > > > Raised a few comments on arch, replaying here
>> > > > > > 1.Do we need to consider upgrading from old arch to new arch, like
>> > > how
>> > > > to
>> > > > > > seamlessly migrate old Griffin tasks into the new one?
>> > > > > > 2.The arch diagram is focusing on service relationships, can you add
>> > > > > > component or deployment diagram to describe it from different angles?
>> > > > > >
>> > > > > > Thx
>> > > > > >
>> > > > > > On Sun, Aug 28, 2022 at 2:31 PM William Guo <gu...@apache.org>
>> > > wrote:
>> > > > > >
>> > > > > > > Hi Eugene,
>> > > > > > >
>> > > > > > > Granted.
>> > > > > > > Everyone can comment on our space.
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > William
>> > > > > > >
>> > > > > > > On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <li...@apache.org>
>> > > > wrote:
>> > > > > > >
>> > > > > > > > Hi William
>> > > > > > > >
>> > > > > > > > Could you grant comment permission on the confluence page? so it
>> > > > > will
>> > > > > > > ease
>> > > > > > > > the conversations of proposal discussion among the Griffin
>> > > > community.
>> > > > > > > >
>> > > > > > > > Thx
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Fri, Aug 26, 2022 at 11:46 AM William Guo <gu...@apache.org>
>> > > > > wrote:
>> > > > > > > >
>> > > > > > > > > hi all,
>> > > > > > > > >
>> > > > > > > > > Some dev might also be interested to review our proposal for a
>> > > > new
>> > > > > > > arch.
>> > > > > > > > > Your thoughts?
>> > > > > > > > >
>> > > > > > > > > Please check this link to review.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > > https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
>> > > > > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > > William
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
Re: [DISCUSS] Griffin new arch
Posted by William Guo <gu...@apache.org>.
hi Willem,
I'm preparing the roadmap in our jira, will finish it tomorrow.
Thanks,
William
On Tue, Oct 11, 2022 at 10:21 AM Willem Jiang <wi...@gmail.com>
wrote:
> It is good to see we are planning a new version of the griffin.
> I think we can write a roadmap to help users and other contributors to
> know better about the whole process.
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Mon, Oct 10, 2022 at 11:52 PM ding wang <wa...@gmail.com> wrote:
> >
> > hi jianhua
> >
> > I would like to join to implement the core module, and since it needs to
> > communicate with dispatcher module.
> >
> > I will provide the communication protocol(interface) between core and
> > dispatcher according to the architecture.
> >
> > Thanks,
> >
> > Wang
> >
> > William Guo <gu...@apache.org> 於 2022年10月10日 週一 晚上7:42寫道：
> >
> > > Cool.
> > >
> > > Before we discuss, could you please document dispatcher design for us
> to
> > > review.
> > >
> > > Thanks,
> > >
> > >
> > >
> > > On Mon, Oct 10, 2022 at 6:20 PM jianhua guo <gu...@gmail.com>
> wrote:
> > >
> > > > hi all,
> > > >
> > > > I have reviewed the new arch, and will try to implement the
> dispatcher
> > > > module in the coming days.
> > > >
> > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> > > >
> > > > https://github.com/apache/griffin/tree/griffin-1.0.0-dev
> > > >
> > > > I need to discuss with you the dispatcher interface with the core
> module.
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Jianhua
> > > >
> > > >
> > > >
> > > > On Wed, Aug 31, 2022 at 6:54 AM William Guo <gu...@apache.org>
> wrote:
> > > >
> > > > > hi Eugene,
> > > > >
> > > > > 1. We need to seamlessly migrate from old tasks to new workflows,
> some
> > > > > modules will be optional in new workflows since old tasks lack
> them.
> > > > > 2. I will draw some diagrams to illustrate it.
> > > > >
> > > > > Thanks,
> > > > > William
> > > > >
> > > > > On Sun, Aug 28, 2022 at 5:07 PM Eugene Law <li...@apache.org>
> wrote:
> > > > >
> > > > > > Hi William
> > > > > >
> > > > > > Raised a few comments on arch, replaying here
> > > > > > 1.Do we need to consider upgrading from old arch to new arch,
> like
> > > how
> > > > to
> > > > > > seamlessly migrate old Griffin tasks into the new one?
> > > > > > 2.The arch diagram is focusing on service relationships, can you
> add
> > > > > > component or deployment diagram to describe it from different
> angles?
> > > > > >
> > > > > > Thx
> > > > > >
> > > > > > On Sun, Aug 28, 2022 at 2:31 PM William Guo <gu...@apache.org>
> > > wrote:
> > > > > >
> > > > > > > Hi Eugene,
> > > > > > >
> > > > > > > Granted.
> > > > > > > Everyone can comment on our space.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > William
> > > > > > >
> > > > > > > On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <liujin@apache.org
> >
> > > > wrote:
> > > > > > >
> > > > > > > > Hi William
> > > > > > > >
> > > > > > > > Could you grant comment permission on the confluence page?
> so it
> > > > > will
> > > > > > > ease
> > > > > > > > the conversations of proposal discussion among the Griffin
> > > > community.
> > > > > > > >
> > > > > > > > Thx
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Aug 26, 2022 at 11:46 AM William Guo <
> guoyp@apache.org>
> > > > > wrote:
> > > > > > > >
> > > > > > > > > hi all,
> > > > > > > > >
> > > > > > > > > Some dev might also be interested to review our proposal
> for a
> > > > new
> > > > > > > arch.
> > > > > > > > > Your thoughts?
> > > > > > > > >
> > > > > > > > > Please check this link to review.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > William
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
Re: [DISCUSS] Griffin new arch
Posted by Willem Jiang <wi...@gmail.com>.
It is good to see we are planning a new version of the griffin.
I think we can write a roadmap to help users and other contributors to
know better about the whole process.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Mon, Oct 10, 2022 at 11:52 PM ding wang <wa...@gmail.com> wrote:
>
> hi jianhua
>
> I would like to join to implement the core module, and since it needs to
> communicate with dispatcher module.
>
> I will provide the communication protocol(interface) between core and
> dispatcher according to the architecture.
>
> Thanks,
>
> Wang
>
> William Guo <gu...@apache.org> 於 2022年10月10日 週一 晚上7:42寫道：
>
> > Cool.
> >
> > Before we discuss, could you please document dispatcher design for us to
> > review.
> >
> > Thanks,
> >
> >
> >
> > On Mon, Oct 10, 2022 at 6:20 PM jianhua guo <gu...@gmail.com> wrote:
> >
> > > hi all,
> > >
> > > I have reviewed the new arch, and will try to implement the dispatcher
> > > module in the coming days.
> > >
> > >
> > >
> > https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> > >
> > > https://github.com/apache/griffin/tree/griffin-1.0.0-dev
> > >
> > > I need to discuss with you the dispatcher interface with the core module.
> > >
> > >
> > > Thanks,
> > >
> > > Jianhua
> > >
> > >
> > >
> > > On Wed, Aug 31, 2022 at 6:54 AM William Guo <gu...@apache.org> wrote:
> > >
> > > > hi Eugene,
> > > >
> > > > 1. We need to seamlessly migrate from old tasks to new workflows, some
> > > > modules will be optional in new workflows since old tasks lack them.
> > > > 2. I will draw some diagrams to illustrate it.
> > > >
> > > > Thanks,
> > > > William
> > > >
> > > > On Sun, Aug 28, 2022 at 5:07 PM Eugene Law <li...@apache.org> wrote:
> > > >
> > > > > Hi William
> > > > >
> > > > > Raised a few comments on arch, replaying here
> > > > > 1.Do we need to consider upgrading from old arch to new arch, like
> > how
> > > to
> > > > > seamlessly migrate old Griffin tasks into the new one?
> > > > > 2.The arch diagram is focusing on service relationships, can you add
> > > > > component or deployment diagram to describe it from different angles?
> > > > >
> > > > > Thx
> > > > >
> > > > > On Sun, Aug 28, 2022 at 2:31 PM William Guo <gu...@apache.org>
> > wrote:
> > > > >
> > > > > > Hi Eugene,
> > > > > >
> > > > > > Granted.
> > > > > > Everyone can comment on our space.
> > > > > >
> > > > > > Thanks,
> > > > > > William
> > > > > >
> > > > > > On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <li...@apache.org>
> > > wrote:
> > > > > >
> > > > > > > Hi William
> > > > > > >
> > > > > > > Could you grant comment permission on the confluence page? so it
> > > > will
> > > > > > ease
> > > > > > > the conversations of proposal discussion among the Griffin
> > > community.
> > > > > > >
> > > > > > > Thx
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Aug 26, 2022 at 11:46 AM William Guo <gu...@apache.org>
> > > > wrote:
> > > > > > >
> > > > > > > > hi all,
> > > > > > > >
> > > > > > > > Some dev might also be interested to review our proposal for a
> > > new
> > > > > > arch.
> > > > > > > > Your thoughts?
> > > > > > > >
> > > > > > > > Please check this link to review.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > William
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
Re: [DISCUSS] Griffin new arch
Posted by ding wang <wa...@gmail.com>.
hi jianhua
I would like to join to implement the core module, and since it needs to
communicate with dispatcher module.
I will provide the communication protocol(interface) between core and
dispatcher according to the architecture.
Thanks,
Wang
William Guo <gu...@apache.org> 於 2022年10月10日 週一 晚上7:42寫道：
> Cool.
>
> Before we discuss, could you please document dispatcher design for us to
> review.
>
> Thanks,
>
>
>
> On Mon, Oct 10, 2022 at 6:20 PM jianhua guo <gu...@gmail.com> wrote:
>
> > hi all,
> >
> > I have reviewed the new arch, and will try to implement the dispatcher
> > module in the coming days.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> >
> > https://github.com/apache/griffin/tree/griffin-1.0.0-dev
> >
> > I need to discuss with you the dispatcher interface with the core module.
> >
> >
> > Thanks,
> >
> > Jianhua
> >
> >
> >
> > On Wed, Aug 31, 2022 at 6:54 AM William Guo <gu...@apache.org> wrote:
> >
> > > hi Eugene,
> > >
> > > 1. We need to seamlessly migrate from old tasks to new workflows, some
> > > modules will be optional in new workflows since old tasks lack them.
> > > 2. I will draw some diagrams to illustrate it.
> > >
> > > Thanks,
> > > William
> > >
> > > On Sun, Aug 28, 2022 at 5:07 PM Eugene Law <li...@apache.org> wrote:
> > >
> > > > Hi William
> > > >
> > > > Raised a few comments on arch, replaying here
> > > > 1.Do we need to consider upgrading from old arch to new arch, like
> how
> > to
> > > > seamlessly migrate old Griffin tasks into the new one?
> > > > 2.The arch diagram is focusing on service relationships, can you add
> > > > component or deployment diagram to describe it from different angles?
> > > >
> > > > Thx
> > > >
> > > > On Sun, Aug 28, 2022 at 2:31 PM William Guo <gu...@apache.org>
> wrote:
> > > >
> > > > > Hi Eugene,
> > > > >
> > > > > Granted.
> > > > > Everyone can comment on our space.
> > > > >
> > > > > Thanks,
> > > > > William
> > > > >
> > > > > On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <li...@apache.org>
> > wrote:
> > > > >
> > > > > > Hi William
> > > > > >
> > > > > > Could you grant comment permission on the confluence page? so it
> > > will
> > > > > ease
> > > > > > the conversations of proposal discussion among the Griffin
> > community.
> > > > > >
> > > > > > Thx
> > > > > >
> > > > > >
> > > > > > On Fri, Aug 26, 2022 at 11:46 AM William Guo <gu...@apache.org>
> > > wrote:
> > > > > >
> > > > > > > hi all,
> > > > > > >
> > > > > > > Some dev might also be interested to review our proposal for a
> > new
> > > > > arch.
> > > > > > > Your thoughts?
> > > > > > >
> > > > > > > Please check this link to review.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> > > > > > >
> > > > > > > Thanks,
> > > > > > > William
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Griffin new arch
Posted by William Guo <gu...@apache.org>.
Cool.
Before we discuss, could you please document dispatcher design for us to
review.
Thanks,
On Mon, Oct 10, 2022 at 6:20 PM jianhua guo <gu...@gmail.com> wrote:
> hi all,
>
> I have reviewed the new arch, and will try to implement the dispatcher
> module in the coming days.
>
>
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
>
> https://github.com/apache/griffin/tree/griffin-1.0.0-dev
>
> I need to discuss with you the dispatcher interface with the core module.
>
>
> Thanks,
>
> Jianhua
>
>
>
> On Wed, Aug 31, 2022 at 6:54 AM William Guo <gu...@apache.org> wrote:
>
> > hi Eugene,
> >
> > 1. We need to seamlessly migrate from old tasks to new workflows, some
> > modules will be optional in new workflows since old tasks lack them.
> > 2. I will draw some diagrams to illustrate it.
> >
> > Thanks,
> > William
> >
> > On Sun, Aug 28, 2022 at 5:07 PM Eugene Law <li...@apache.org> wrote:
> >
> > > Hi William
> > >
> > > Raised a few comments on arch, replaying here
> > > 1.Do we need to consider upgrading from old arch to new arch, like how
> to
> > > seamlessly migrate old Griffin tasks into the new one?
> > > 2.The arch diagram is focusing on service relationships, can you add
> > > component or deployment diagram to describe it from different angles?
> > >
> > > Thx
> > >
> > > On Sun, Aug 28, 2022 at 2:31 PM William Guo <gu...@apache.org> wrote:
> > >
> > > > Hi Eugene,
> > > >
> > > > Granted.
> > > > Everyone can comment on our space.
> > > >
> > > > Thanks,
> > > > William
> > > >
> > > > On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <li...@apache.org>
> wrote:
> > > >
> > > > > Hi William
> > > > >
> > > > > Could you grant comment permission on the confluence page? so it
> > will
> > > > ease
> > > > > the conversations of proposal discussion among the Griffin
> community.
> > > > >
> > > > > Thx
> > > > >
> > > > >
> > > > > On Fri, Aug 26, 2022 at 11:46 AM William Guo <gu...@apache.org>
> > wrote:
> > > > >
> > > > > > hi all,
> > > > > >
> > > > > > Some dev might also be interested to review our proposal for a
> new
> > > > arch.
> > > > > > Your thoughts?
> > > > > >
> > > > > > Please check this link to review.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> > > > > >
> > > > > > Thanks,
> > > > > > William
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Griffin new arch
Posted by jianhua guo <gu...@gmail.com>.
hi all,
I have reviewed the new arch, and will try to implement the dispatcher
module in the coming days.
https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
https://github.com/apache/griffin/tree/griffin-1.0.0-dev
I need to discuss with you the dispatcher interface with the core module.
Thanks,
Jianhua
On Wed, Aug 31, 2022 at 6:54 AM William Guo <gu...@apache.org> wrote:
> hi Eugene,
>
> 1. We need to seamlessly migrate from old tasks to new workflows, some
> modules will be optional in new workflows since old tasks lack them.
> 2. I will draw some diagrams to illustrate it.
>
> Thanks,
> William
>
> On Sun, Aug 28, 2022 at 5:07 PM Eugene Law <li...@apache.org> wrote:
>
> > Hi William
> >
> > Raised a few comments on arch, replaying here
> > 1.Do we need to consider upgrading from old arch to new arch, like how to
> > seamlessly migrate old Griffin tasks into the new one?
> > 2.The arch diagram is focusing on service relationships, can you add
> > component or deployment diagram to describe it from different angles?
> >
> > Thx
> >
> > On Sun, Aug 28, 2022 at 2:31 PM William Guo <gu...@apache.org> wrote:
> >
> > > Hi Eugene,
> > >
> > > Granted.
> > > Everyone can comment on our space.
> > >
> > > Thanks,
> > > William
> > >
> > > On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <li...@apache.org> wrote:
> > >
> > > > Hi William
> > > >
> > > > Could you grant comment permission on the confluence page? so it
> will
> > > ease
> > > > the conversations of proposal discussion among the Griffin community.
> > > >
> > > > Thx
> > > >
> > > >
> > > > On Fri, Aug 26, 2022 at 11:46 AM William Guo <gu...@apache.org>
> wrote:
> > > >
> > > > > hi all,
> > > > >
> > > > > Some dev might also be interested to review our proposal for a new
> > > arch.
> > > > > Your thoughts?
> > > > >
> > > > > Please check this link to review.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> > > > >
> > > > > Thanks,
> > > > > William
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Griffin new arch
Posted by William Guo <gu...@apache.org>.
hi Eugene,
1. We need to seamlessly migrate from old tasks to new workflows, some
modules will be optional in new workflows since old tasks lack them.
2. I will draw some diagrams to illustrate it.
Thanks,
William
On Sun, Aug 28, 2022 at 5:07 PM Eugene Law <li...@apache.org> wrote:
> Hi William
>
> Raised a few comments on arch, replaying here
> 1.Do we need to consider upgrading from old arch to new arch, like how to
> seamlessly migrate old Griffin tasks into the new one?
> 2.The arch diagram is focusing on service relationships, can you add
> component or deployment diagram to describe it from different angles?
>
> Thx
>
> On Sun, Aug 28, 2022 at 2:31 PM William Guo <gu...@apache.org> wrote:
>
> > Hi Eugene,
> >
> > Granted.
> > Everyone can comment on our space.
> >
> > Thanks,
> > William
> >
> > On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <li...@apache.org> wrote:
> >
> > > Hi William
> > >
> > > Could you grant comment permission on the confluence page? so it will
> > ease
> > > the conversations of proposal discussion among the Griffin community.
> > >
> > > Thx
> > >
> > >
> > > On Fri, Aug 26, 2022 at 11:46 AM William Guo <gu...@apache.org> wrote:
> > >
> > > > hi all,
> > > >
> > > > Some dev might also be interested to review our proposal for a new
> > arch.
> > > > Your thoughts?
> > > >
> > > > Please check this link to review.
> > > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> > > >
> > > > Thanks,
> > > > William
> > > >
> > >
> >
>
Re: [DISCUSS] Griffin new arch
Posted by Eugene Law <li...@apache.org>.
Hi William
Raised a few comments on arch, replaying here
1.Do we need to consider upgrading from old arch to new arch, like how to
seamlessly migrate old Griffin tasks into the new one?
2.The arch diagram is focusing on service relationships, can you add
component or deployment diagram to describe it from different angles?
Thx
On Sun, Aug 28, 2022 at 2:31 PM William Guo <gu...@apache.org> wrote:
> Hi Eugene,
>
> Granted.
> Everyone can comment on our space.
>
> Thanks,
> William
>
> On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <li...@apache.org> wrote:
>
> > Hi William
> >
> > Could you grant comment permission on the confluence page? so it will
> ease
> > the conversations of proposal discussion among the Griffin community.
> >
> > Thx
> >
> >
> > On Fri, Aug 26, 2022 at 11:46 AM William Guo <gu...@apache.org> wrote:
> >
> > > hi all,
> > >
> > > Some dev might also be interested to review our proposal for a new
> arch.
> > > Your thoughts?
> > >
> > > Please check this link to review.
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> > >
> > > Thanks,
> > > William
> > >
> >
>
Re: [DISCUSS] Griffin new arch
Posted by William Guo <gu...@apache.org>.
Hi Eugene,
Granted.
Everyone can comment on our space.
Thanks,
William
On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <li...@apache.org> wrote:
> Hi William
>
> Could you grant comment permission on the confluence page? so it will ease
> the conversations of proposal discussion among the Griffin community.
>
> Thx
>
>
> On Fri, Aug 26, 2022 at 11:46 AM William Guo <gu...@apache.org> wrote:
>
> > hi all,
> >
> > Some dev might also be interested to review our proposal for a new arch.
> > Your thoughts?
> >
> > Please check this link to review.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> >
> > Thanks,
> > William
> >
>
Re: [DISCUSS] Griffin new arch
Posted by Eugene Law <li...@apache.org>.
Hi William
Could you grant comment permission on the confluence page? so it will ease
the conversations of proposal discussion among the Griffin community.
Thx
On Fri, Aug 26, 2022 at 11:46 AM William Guo <gu...@apache.org> wrote:
> hi all,
>
> Some dev might also be interested to review our proposal for a new arch.
> Your thoughts?
>
> Please check this link to review.
>
>
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
>
> Thanks,
> William
>