You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@servicecomb.apache.org by zhaojun <zh...@126.com> on 2019/04/26 10:39:38 UTC
[saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Hi, all
currently, we have integrated with saga using graph based engine in shardingsphere[1]
it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
so recovery thread will not be executed correctly.
it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
any thought?
[1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Willem Jiang <wi...@gmail.com>.
I think it's a quick different story, as Skywalking agent doesn't
invoke the SQL execution.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Fri, Apr 26, 2019 at 6:47 PM Sheng Wu <wu...@apache.org> wrote:
>
> Please be advised, SkyWalking and seata integration are ongoing too.
>
> https://github.com/seata/seata/issues/714
>
> SkyWalking agent even will be binding in Seata TC binary release.
>
> As shardingsphere pmc are developing SkyWalking plugin too. Please be
> careful, don't make the conflict.
>
> zhaojun <zh...@126.com>于2019年4月26日 周五下午6:41写道：
>
> > Hi, all
> >
> > currently, we have integrated with saga using graph based engine in
> > shardingsphere[1]
> > it need us to collect all participated actual SQL, then submit to saga
> > actuator in commit/rollback phase.
> > if application crashed before invoking saga actuator, undo log of branch
> > transaction SQL will not be saved,
> > so recovery thread will not be executed correctly.
> >
> > it's better that encapsulating every actual SQL as a saga task in
> > shardingsphere side,
> > then submit to saga actuator realtime instead of batch processing all the
> > SQLs at commit/rollback phase.
> > this architecture will make the boundary more clear between shardingsphre
> > and saga, currently we have done some additional works for integrating saga.
> >
> > any thought?
> >
> > [1]:
> > https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> > <
> > https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> > >
> >
> >
> >
> > ------------------
> > Zhao Jun
> > Apache Sharding-Sphere & ServiceComb
> >
> > --
> Sheng Wu
> SkyWalking, Shardingsphere and Zipkin
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by zhaojun <zh...@126.com>.
Hi, zheng, Sheng
Revert SQL have been in compensate method method automatically. We have completed this feature.
You can see more detail from our example[1].
Also we have some issue [2] to do. shardingsphere-transaction-base-saga could be using in cloud native environment. I think this model is more effective.
[1] https://github.com/apache/incubator-shardingsphere-example/tree/dev/sharding-jdbc-example/transaction-example/transaction-base-saga-example <https://github.com/apache/incubator-shardingsphere-example/tree/dev/sharding-jdbc-example/transaction-example/transaction-base-saga-example>
[2]https://github.com/sharding-sphere/shardingsphere-spi-impl/issues?q=is%3Aopen+is%3Aissue+label%3Asaga <https://github.com/sharding-sphere/shardingsphere-spi-impl/issues?q=is:open+is:issue+label:saga>
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
> On Apr 26, 2019, at 7:16 PM, Sheng Wu <wu...@gmail.com> wrote:
>
> Hi
>
> SkyWalking will do agent based auto instrumentation by runtime code
> manipulate tech. So we would depend on SPI or particular interface.
> SkyWalking plans to intercept some core APIs.
>
> ServiceComb could be supported in same way.
>
> I cc'd dev@skywalking
>
> Zhenxu Ke and Yuguang Zhao, you could share your design to Shardingsphere
> community too.
>
> Zheng Feng <zh...@gmail.com>于2019年4月26日 周五下午7:02写道：
>
>> Thanks Sheng for pointing out this and I just wonder the SkyWalking will
>> trace the transaction event according to the issue [1]. And this
>> transaction event could have the common SPI interface with the different
>> vendor implementation, e.g. the ServiceComb and the Seata ?
>>
>> Thanks,
>> Zheng Feng
>>
>> [1] https://github.com/seata/seata/issues/714
>>
>> Sheng Wu <wu...@apache.org> 于2019年4月26日周五 下午6:47写道：
>>
>>> Please be advised, SkyWalking and seata integration are ongoing too.
>>>
>>> https://github.com/seata/seata/issues/714
>>>
>>> SkyWalking agent even will be binding in Seata TC binary release.
>>>
>>> As shardingsphere pmc are developing SkyWalking plugin too. Please be
>>> careful, don't make the conflict.
>>>
>>> zhaojun <zh...@126.com>于2019年4月26日 周五下午6:41写道：
>>>
>>>> Hi, all
>>>>
>>>> currently, we have integrated with saga using graph based engine in
>>>> shardingsphere[1]
>>>> it need us to collect all participated actual SQL, then submit to saga
>>>> actuator in commit/rollback phase.
>>>> if application crashed before invoking saga actuator, undo log of
>> branch
>>>> transaction SQL will not be saved,
>>>> so recovery thread will not be executed correctly.
>>>>
>>>> it's better that encapsulating every actual SQL as a saga task in
>>>> shardingsphere side,
>>>> then submit to saga actuator realtime instead of batch processing all
>> the
>>>> SQLs at commit/rollback phase.
>>>> this architecture will make the boundary more clear between
>> shardingsphre
>>>> and saga, currently we have done some additional works for integrating
>>> saga.
>>>>
>>>> any thought?
>>>>
>>>> [1]:
>>>>
>>>
>> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
>>>> <
>>>>
>>>
>> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
>>>>>
>>>>
>>>>
>>>>
>>>> ------------------
>>>> Zhao Jun
>>>> Apache Sharding-Sphere & ServiceComb
>>>>
>>>> --
>>> Sheng Wu
>>> SkyWalking, Shardingsphere and Zipkin
>>>
>>
> --
> Sheng Wu
> SkyWalking, Shardingsphere and Zipkin
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by zhaojun <zh...@126.com>.
Hi, zheng, Sheng
Revert SQL have been in compensate method method automatically. We have completed this feature.
You can see more detail from our example[1].
Also we have some issue [2] to do. shardingsphere-transaction-base-saga could be using in cloud native environment. I think this model is more effective.
[1] https://github.com/apache/incubator-shardingsphere-example/tree/dev/sharding-jdbc-example/transaction-example/transaction-base-saga-example <https://github.com/apache/incubator-shardingsphere-example/tree/dev/sharding-jdbc-example/transaction-example/transaction-base-saga-example>
[2]https://github.com/sharding-sphere/shardingsphere-spi-impl/issues?q=is%3Aopen+is%3Aissue+label%3Asaga <https://github.com/sharding-sphere/shardingsphere-spi-impl/issues?q=is:open+is:issue+label:saga>
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
> On Apr 26, 2019, at 7:16 PM, Sheng Wu <wu...@gmail.com> wrote:
>
> Hi
>
> SkyWalking will do agent based auto instrumentation by runtime code
> manipulate tech. So we would depend on SPI or particular interface.
> SkyWalking plans to intercept some core APIs.
>
> ServiceComb could be supported in same way.
>
> I cc'd dev@skywalking
>
> Zhenxu Ke and Yuguang Zhao, you could share your design to Shardingsphere
> community too.
>
> Zheng Feng <zh...@gmail.com>于2019年4月26日 周五下午7:02写道：
>
>> Thanks Sheng for pointing out this and I just wonder the SkyWalking will
>> trace the transaction event according to the issue [1]. And this
>> transaction event could have the common SPI interface with the different
>> vendor implementation, e.g. the ServiceComb and the Seata ?
>>
>> Thanks,
>> Zheng Feng
>>
>> [1] https://github.com/seata/seata/issues/714
>>
>> Sheng Wu <wu...@apache.org> 于2019年4月26日周五 下午6:47写道：
>>
>>> Please be advised, SkyWalking and seata integration are ongoing too.
>>>
>>> https://github.com/seata/seata/issues/714
>>>
>>> SkyWalking agent even will be binding in Seata TC binary release.
>>>
>>> As shardingsphere pmc are developing SkyWalking plugin too. Please be
>>> careful, don't make the conflict.
>>>
>>> zhaojun <zh...@126.com>于2019年4月26日 周五下午6:41写道：
>>>
>>>> Hi, all
>>>>
>>>> currently, we have integrated with saga using graph based engine in
>>>> shardingsphere[1]
>>>> it need us to collect all participated actual SQL, then submit to saga
>>>> actuator in commit/rollback phase.
>>>> if application crashed before invoking saga actuator, undo log of
>> branch
>>>> transaction SQL will not be saved,
>>>> so recovery thread will not be executed correctly.
>>>>
>>>> it's better that encapsulating every actual SQL as a saga task in
>>>> shardingsphere side,
>>>> then submit to saga actuator realtime instead of batch processing all
>> the
>>>> SQLs at commit/rollback phase.
>>>> this architecture will make the boundary more clear between
>> shardingsphre
>>>> and saga, currently we have done some additional works for integrating
>>> saga.
>>>>
>>>> any thought?
>>>>
>>>> [1]:
>>>>
>>>
>> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
>>>> <
>>>>
>>>
>> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
>>>>>
>>>>
>>>>
>>>>
>>>> ------------------
>>>> Zhao Jun
>>>> Apache Sharding-Sphere & ServiceComb
>>>>
>>>> --
>>> Sheng Wu
>>> SkyWalking, Shardingsphere and Zipkin
>>>
>>
> --
> Sheng Wu
> SkyWalking, Shardingsphere and Zipkin
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by zhaojun <zh...@126.com>.
Hi, zheng, Sheng
Revert SQL have been in compensate method method automatically. We have completed this feature.
You can see more detail from our example[1].
Also we have some issue [2] to do. shardingsphere-transaction-base-saga could be using in cloud native environment. I think this model is more effective.
[1] https://github.com/apache/incubator-shardingsphere-example/tree/dev/sharding-jdbc-example/transaction-example/transaction-base-saga-example <https://github.com/apache/incubator-shardingsphere-example/tree/dev/sharding-jdbc-example/transaction-example/transaction-base-saga-example>
[2]https://github.com/sharding-sphere/shardingsphere-spi-impl/issues?q=is%3Aopen+is%3Aissue+label%3Asaga <https://github.com/sharding-sphere/shardingsphere-spi-impl/issues?q=is:open+is:issue+label:saga>
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
> On Apr 26, 2019, at 7:16 PM, Sheng Wu <wu...@gmail.com> wrote:
>
> Hi
>
> SkyWalking will do agent based auto instrumentation by runtime code
> manipulate tech. So we would depend on SPI or particular interface.
> SkyWalking plans to intercept some core APIs.
>
> ServiceComb could be supported in same way.
>
> I cc'd dev@skywalking
>
> Zhenxu Ke and Yuguang Zhao, you could share your design to Shardingsphere
> community too.
>
> Zheng Feng <zh...@gmail.com>于2019年4月26日 周五下午7:02写道：
>
>> Thanks Sheng for pointing out this and I just wonder the SkyWalking will
>> trace the transaction event according to the issue [1]. And this
>> transaction event could have the common SPI interface with the different
>> vendor implementation, e.g. the ServiceComb and the Seata ?
>>
>> Thanks,
>> Zheng Feng
>>
>> [1] https://github.com/seata/seata/issues/714
>>
>> Sheng Wu <wu...@apache.org> 于2019年4月26日周五 下午6:47写道：
>>
>>> Please be advised, SkyWalking and seata integration are ongoing too.
>>>
>>> https://github.com/seata/seata/issues/714
>>>
>>> SkyWalking agent even will be binding in Seata TC binary release.
>>>
>>> As shardingsphere pmc are developing SkyWalking plugin too. Please be
>>> careful, don't make the conflict.
>>>
>>> zhaojun <zh...@126.com>于2019年4月26日 周五下午6:41写道：
>>>
>>>> Hi, all
>>>>
>>>> currently, we have integrated with saga using graph based engine in
>>>> shardingsphere[1]
>>>> it need us to collect all participated actual SQL, then submit to saga
>>>> actuator in commit/rollback phase.
>>>> if application crashed before invoking saga actuator, undo log of
>> branch
>>>> transaction SQL will not be saved,
>>>> so recovery thread will not be executed correctly.
>>>>
>>>> it's better that encapsulating every actual SQL as a saga task in
>>>> shardingsphere side,
>>>> then submit to saga actuator realtime instead of batch processing all
>> the
>>>> SQLs at commit/rollback phase.
>>>> this architecture will make the boundary more clear between
>> shardingsphre
>>>> and saga, currently we have done some additional works for integrating
>>> saga.
>>>>
>>>> any thought?
>>>>
>>>> [1]:
>>>>
>>>
>> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
>>>> <
>>>>
>>>
>> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
>>>>>
>>>>
>>>>
>>>>
>>>> ------------------
>>>> Zhao Jun
>>>> Apache Sharding-Sphere & ServiceComb
>>>>
>>>> --
>>> Sheng Wu
>>> SkyWalking, Shardingsphere and Zipkin
>>>
>>
> --
> Sheng Wu
> SkyWalking, Shardingsphere and Zipkin
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Sheng Wu <wu...@gmail.com>.
Hi
SkyWalking will do agent based auto instrumentation by runtime code
manipulate tech. So we would depend on SPI or particular interface.
SkyWalking plans to intercept some core APIs.
ServiceComb could be supported in same way.
I cc'd dev@skywalking
Zhenxu Ke and Yuguang Zhao, you could share your design to Shardingsphere
community too.
Zheng Feng <zh...@gmail.com>于2019年4月26日 周五下午7:02写道：
> Thanks Sheng for pointing out this and I just wonder the SkyWalking will
> trace the transaction event according to the issue [1]. And this
> transaction event could have the common SPI interface with the different
> vendor implementation, e.g. the ServiceComb and the Seata ?
>
> Thanks,
> Zheng Feng
>
> [1] https://github.com/seata/seata/issues/714
>
> Sheng Wu <wu...@apache.org> 于2019年4月26日周五 下午6:47写道：
>
> > Please be advised, SkyWalking and seata integration are ongoing too.
> >
> > https://github.com/seata/seata/issues/714
> >
> > SkyWalking agent even will be binding in Seata TC binary release.
> >
> > As shardingsphere pmc are developing SkyWalking plugin too. Please be
> > careful, don't make the conflict.
> >
> > zhaojun <zh...@126.com>于2019年4月26日 周五下午6:41写道：
> >
> > > Hi, all
> > >
> > > currently, we have integrated with saga using graph based engine in
> > > shardingsphere[1]
> > > it need us to collect all participated actual SQL, then submit to saga
> > > actuator in commit/rollback phase.
> > > if application crashed before invoking saga actuator, undo log of
> branch
> > > transaction SQL will not be saved,
> > > so recovery thread will not be executed correctly.
> > >
> > > it's better that encapsulating every actual SQL as a saga task in
> > > shardingsphere side,
> > > then submit to saga actuator realtime instead of batch processing all
> the
> > > SQLs at commit/rollback phase.
> > > this architecture will make the boundary more clear between
> shardingsphre
> > > and saga, currently we have done some additional works for integrating
> > saga.
> > >
> > > any thought?
> > >
> > > [1]:
> > >
> >
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> > > <
> > >
> >
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> > > >
> > >
> > >
> > >
> > > ------------------
> > > Zhao Jun
> > > Apache Sharding-Sphere & ServiceComb
> > >
> > > --
> > Sheng Wu
> > SkyWalking, Shardingsphere and Zipkin
> >
>
--
Sheng Wu
SkyWalking, Shardingsphere and Zipkin
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Sheng Wu <wu...@gmail.com>.
Hi
SkyWalking will do agent based auto instrumentation by runtime code
manipulate tech. So we would depend on SPI or particular interface.
SkyWalking plans to intercept some core APIs.
ServiceComb could be supported in same way.
I cc'd dev@skywalking
Zhenxu Ke and Yuguang Zhao, you could share your design to Shardingsphere
community too.
Zheng Feng <zh...@gmail.com>于2019年4月26日 周五下午7:02写道：
> Thanks Sheng for pointing out this and I just wonder the SkyWalking will
> trace the transaction event according to the issue [1]. And this
> transaction event could have the common SPI interface with the different
> vendor implementation, e.g. the ServiceComb and the Seata ?
>
> Thanks,
> Zheng Feng
>
> [1] https://github.com/seata/seata/issues/714
>
> Sheng Wu <wu...@apache.org> 于2019年4月26日周五 下午6:47写道：
>
> > Please be advised, SkyWalking and seata integration are ongoing too.
> >
> > https://github.com/seata/seata/issues/714
> >
> > SkyWalking agent even will be binding in Seata TC binary release.
> >
> > As shardingsphere pmc are developing SkyWalking plugin too. Please be
> > careful, don't make the conflict.
> >
> > zhaojun <zh...@126.com>于2019年4月26日 周五下午6:41写道：
> >
> > > Hi, all
> > >
> > > currently, we have integrated with saga using graph based engine in
> > > shardingsphere[1]
> > > it need us to collect all participated actual SQL, then submit to saga
> > > actuator in commit/rollback phase.
> > > if application crashed before invoking saga actuator, undo log of
> branch
> > > transaction SQL will not be saved,
> > > so recovery thread will not be executed correctly.
> > >
> > > it's better that encapsulating every actual SQL as a saga task in
> > > shardingsphere side,
> > > then submit to saga actuator realtime instead of batch processing all
> the
> > > SQLs at commit/rollback phase.
> > > this architecture will make the boundary more clear between
> shardingsphre
> > > and saga, currently we have done some additional works for integrating
> > saga.
> > >
> > > any thought?
> > >
> > > [1]:
> > >
> >
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> > > <
> > >
> >
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> > > >
> > >
> > >
> > >
> > > ------------------
> > > Zhao Jun
> > > Apache Sharding-Sphere & ServiceComb
> > >
> > > --
> > Sheng Wu
> > SkyWalking, Shardingsphere and Zipkin
> >
>
--
Sheng Wu
SkyWalking, Shardingsphere and Zipkin
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Zheng Feng <zh...@gmail.com>.
Thanks Sheng for pointing out this and I just wonder the SkyWalking will
trace the transaction event according to the issue [1]. And this
transaction event could have the common SPI interface with the different
vendor implementation, e.g. the ServiceComb and the Seata ?
Thanks,
Zheng Feng
[1] https://github.com/seata/seata/issues/714
Sheng Wu <wu...@apache.org> 于2019年4月26日周五 下午6:47写道：
> Please be advised, SkyWalking and seata integration are ongoing too.
>
> https://github.com/seata/seata/issues/714
>
> SkyWalking agent even will be binding in Seata TC binary release.
>
> As shardingsphere pmc are developing SkyWalking plugin too. Please be
> careful, don't make the conflict.
>
> zhaojun <zh...@126.com>于2019年4月26日 周五下午6:41写道：
>
> > Hi, all
> >
> > currently, we have integrated with saga using graph based engine in
> > shardingsphere[1]
> > it need us to collect all participated actual SQL, then submit to saga
> > actuator in commit/rollback phase.
> > if application crashed before invoking saga actuator, undo log of branch
> > transaction SQL will not be saved,
> > so recovery thread will not be executed correctly.
> >
> > it's better that encapsulating every actual SQL as a saga task in
> > shardingsphere side,
> > then submit to saga actuator realtime instead of batch processing all the
> > SQLs at commit/rollback phase.
> > this architecture will make the boundary more clear between shardingsphre
> > and saga, currently we have done some additional works for integrating
> saga.
> >
> > any thought?
> >
> > [1]:
> >
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> > <
> >
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> > >
> >
> >
> >
> > ------------------
> > Zhao Jun
> > Apache Sharding-Sphere & ServiceComb
> >
> > --
> Sheng Wu
> SkyWalking, Shardingsphere and Zipkin
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Sheng Wu <wu...@apache.org>.
Please be advised, SkyWalking and seata integration are ongoing too.
https://github.com/seata/seata/issues/714
SkyWalking agent even will be binding in Seata TC binary release.
As shardingsphere pmc are developing SkyWalking plugin too. Please be
careful, don't make the conflict.
zhaojun <zh...@126.com>于2019年4月26日 周五下午6:41写道：
> Hi, all
>
> currently, we have integrated with saga using graph based engine in
> shardingsphere[1]
> it need us to collect all participated actual SQL, then submit to saga
> actuator in commit/rollback phase.
> if application crashed before invoking saga actuator, undo log of branch
> transaction SQL will not be saved,
> so recovery thread will not be executed correctly.
>
> it's better that encapsulating every actual SQL as a saga task in
> shardingsphere side,
> then submit to saga actuator realtime instead of batch processing all the
> SQLs at commit/rollback phase.
> this architecture will make the boundary more clear between shardingsphre
> and saga, currently we have done some additional works for integrating saga.
>
> any thought?
>
> [1]:
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> <
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> >
>
>
>
> ------------------
> Zhao Jun
> Apache Sharding-Sphere & ServiceComb
>
> --
Sheng Wu
SkyWalking, Shardingsphere and Zipkin
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Sheng Wu <wu...@gmail.com>.
Feedback from our Seata and SkyWalking integration.
We will provide plugins for there TC and TM.
Also, Seata team has the plan to distribute their binary including
SkyWalking binary tar in TC, for user convenience.
Then users could use a single command in TC to open SkyWalking agent
working with the globally distributed tracing system, and benefits from
SkyWalking other monitoring/analysis capabilities.
Sheng Wu 吴晟
Apache SkyWalking, ShardingSphere, Zipkin
Twitter, wusheng1108
Willem Jiang <wi...@gmail.com> 于2019年5月1日周三 上午8:44写道：
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
> >
> > Hi, Willem
> >
> > Saga actuator could not compatible with ShardingSphere currently, there
> are 2 main problem exist.
> > 1. If sql execute in saga transport instead of ShardingSphere, we can
> not see the result of logic-sql even in one transaction, it is like this:
> > update t_order set status=’start’ where usr_id=’tom’;
> > select status from t_order where usr_id=’tom’;
> —> we can’t query ’start’ record as actuator haven’t executed.
> > Insert into t_order values(?,?,?) ;
>
> I think we can break the whole SQL interactions into smaller steps,
> and let saga-actuator do the job one by one.
> In this case, you can break the whole calling graph into several sub
> calling graphs.
> Any thought?
>
> > 2. If logic-sql execute in ShardingSphere, we cannot handle recovery
> before submit graph result, as event log only wrote at saga engine.
> >
> > so we should integrate saga-transaction like omega send event log step
> by step.
> > It is better to make every instance do recovery Independently, instead
> of providing another coordinator center service.
> > I feel that embed saga should have following capability.
> > 1).It can provide service in jar package independent
> > 2).each embed saga only recovery their own transaction-data of this
> instance.
> > If one instance crashed, we can introduce external service to do
> failover.
> >
> > so we consider about exending saga-acutator, if it can support submit
> task step by step in one transaction, it is a good choice.
> > of course, there have many things we should do.
> >
> > ------------------
> > Zhao Jun
> > Apache Sharding-Sphere & ServiceComb
> >
> > > On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com>
> wrote:
> > >
> > > First Saga actuator need to build up the calling grapha before sending
> > > out the request. I don't think you can do the step by step SQL
> > > invocation with Saga actuator.
> > > If you want to call the SQL execution step by step , you may need to
> > > switch to ServiceComb Pack project which has a coordinator to take
> > > care of the distributed transaction. But that introduce another
> > > endpoint(Alpha) to shardingsphere.
> > >
> > > From my understanding, Saga actuator is most efficient way to execute
> > > the SQL across different data nodes.
> > >
> > >
> > > Willem Jiang
> > >
> > > Twitter: willemjiang
> > > Weibo: 姜宁willem
> > >
> > > On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
> > >>
> > >> Hi, all
> > >>
> > >> currently, we have integrated with saga using graph based engine in
> shardingsphere[1]
> > >> it need us to collect all participated actual SQL, then submit to
> saga actuator in commit/rollback phase.
> > >> if application crashed before invoking saga actuator, undo log of
> branch transaction SQL will not be saved,
> > >> so recovery thread will not be executed correctly.
> > >>
> > >> it's better that encapsulating every actual SQL as a saga task in
> shardingsphere side,
> > >> then submit to saga actuator realtime instead of batch processing all
> the SQLs at commit/rollback phase.
> > >> this architecture will make the boundary more clear between
> shardingsphre and saga, currently we have done some additional works for
> integrating saga.
> > >>
> > >> any thought?
> > >>
> > >> [1]:
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> <
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> >
> > >>
> > >>
> > >>
> > >> ------------------
> > >> Zhao Jun
> > >> Apache Sharding-Sphere & ServiceComb
> > >>
> >
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Sheng Wu <wu...@gmail.com>.
Feedback from our Seata and SkyWalking integration.
We will provide plugins for there TC and TM.
Also, Seata team has the plan to distribute their binary including
SkyWalking binary tar in TC, for user convenience.
Then users could use a single command in TC to open SkyWalking agent
working with the globally distributed tracing system, and benefits from
SkyWalking other monitoring/analysis capabilities.
Sheng Wu 吴晟
Apache SkyWalking, ShardingSphere, Zipkin
Twitter, wusheng1108
Willem Jiang <wi...@gmail.com> 于2019年5月1日周三 上午8:44写道：
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
> >
> > Hi, Willem
> >
> > Saga actuator could not compatible with ShardingSphere currently, there
> are 2 main problem exist.
> > 1. If sql execute in saga transport instead of ShardingSphere, we can
> not see the result of logic-sql even in one transaction, it is like this:
> > update t_order set status=’start’ where usr_id=’tom’;
> > select status from t_order where usr_id=’tom’;
> —> we can’t query ’start’ record as actuator haven’t executed.
> > Insert into t_order values(?,?,?) ;
>
> I think we can break the whole SQL interactions into smaller steps,
> and let saga-actuator do the job one by one.
> In this case, you can break the whole calling graph into several sub
> calling graphs.
> Any thought?
>
> > 2. If logic-sql execute in ShardingSphere, we cannot handle recovery
> before submit graph result, as event log only wrote at saga engine.
> >
> > so we should integrate saga-transaction like omega send event log step
> by step.
> > It is better to make every instance do recovery Independently, instead
> of providing another coordinator center service.
> > I feel that embed saga should have following capability.
> > 1).It can provide service in jar package independent
> > 2).each embed saga only recovery their own transaction-data of this
> instance.
> > If one instance crashed, we can introduce external service to do
> failover.
> >
> > so we consider about exending saga-acutator, if it can support submit
> task step by step in one transaction, it is a good choice.
> > of course, there have many things we should do.
> >
> > ------------------
> > Zhao Jun
> > Apache Sharding-Sphere & ServiceComb
> >
> > > On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com>
> wrote:
> > >
> > > First Saga actuator need to build up the calling grapha before sending
> > > out the request. I don't think you can do the step by step SQL
> > > invocation with Saga actuator.
> > > If you want to call the SQL execution step by step , you may need to
> > > switch to ServiceComb Pack project which has a coordinator to take
> > > care of the distributed transaction. But that introduce another
> > > endpoint(Alpha) to shardingsphere.
> > >
> > > From my understanding, Saga actuator is most efficient way to execute
> > > the SQL across different data nodes.
> > >
> > >
> > > Willem Jiang
> > >
> > > Twitter: willemjiang
> > > Weibo: 姜宁willem
> > >
> > > On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
> > >>
> > >> Hi, all
> > >>
> > >> currently, we have integrated with saga using graph based engine in
> shardingsphere[1]
> > >> it need us to collect all participated actual SQL, then submit to
> saga actuator in commit/rollback phase.
> > >> if application crashed before invoking saga actuator, undo log of
> branch transaction SQL will not be saved,
> > >> so recovery thread will not be executed correctly.
> > >>
> > >> it's better that encapsulating every actual SQL as a saga task in
> shardingsphere side,
> > >> then submit to saga actuator realtime instead of batch processing all
> the SQLs at commit/rollback phase.
> > >> this architecture will make the boundary more clear between
> shardingsphre and saga, currently we have done some additional works for
> integrating saga.
> > >>
> > >> any thought?
> > >>
> > >> [1]:
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> <
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> >
> > >>
> > >>
> > >>
> > >> ------------------
> > >> Zhao Jun
> > >> Apache Sharding-Sphere & ServiceComb
> > >>
> >
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by zhaojun <zh...@126.com>.
I have just created a issue[1] in JIRA
[1]: https://issues.apache.org/jira/browse/SCB-1276 <https://issues.apache.org/jira/browse/SCB-1276>
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
> On May 5, 2019, at 8:46 AM, zhaojun <zh...@126.com> wrote:
>
> OK, I’ll try to write a JIRA today.
>
> ------------------
> Zhao Jun
> Apache Sharding-Sphere & ServiceComb
>
>
>> On May 2, 2019, at 3:18 PM, Willem Jiang <wi...@gmail.com> wrote:
>>
>> It sounds good to me.
>>
>> Willem Jiang
>>
>> Twitter: willemjiang
>> Weibo: 姜宁willem
>>
>> On Thu, May 2, 2019 at 11:14 AM 赵俊 <zh...@jd.com> wrote:
>>>
>>> Let SQL executor running command one by one is good to me.
>>>
>>> One logic-sql will be routed to multi-actual SQLs, then executed in sharding-sphere execute-engine.
>>> A actual SQL will be wrapped as a SQL execute-callback in sharding-sphere.
>>> We could provide a SPI for this SQL execute-callback.
>>> For integrated with saga, we could implement a saga execute-callback, it will wrap event sourcing logic like saga-actuator task have done.
>>>
>>>
>>>> On May 1, 2019, at 5:06 PM, Willem Jiang <wi...@gmail.com> wrote:
>>>>
>>>> We could do some enhancement on it.
>>>> From my understanding, We could create a calling graph which has three
>>>> sub graph of it, or we could let the SQL executor running the command
>>>> one by one.
>>>> Any thoughts?
>>>>
>>>> Willem Jiang
>>>>
>>>> Twitter: willemjiang
>>>> Weibo: 姜宁willem
>>>>
>>>> On Wed, May 1, 2019 at 4:29 PM zhaojun <zh...@126.com> wrote:
>>>>>
>>>>> But saga-actuator don’t support 3 logic SQL is a global transaction while you submit 3 times separately.
>>>>>
>>>>> ------------------
>>>>> Zhao Jun
>>>>> Apache Sharding-Sphere & ServiceComb
>>>>>
>>>>>> On May 1, 2019, at 8:38 AM, Willem Jiang <wi...@gmail.com> wrote:
>>>>>>
>>>>>> Willem Jiang
>>>>>>
>>>>>> Twitter: willemjiang
>>>>>> Weibo: 姜宁willem
>>>>>>
>>>>>> On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
>>>>>>>
>>>>>>> Hi, Willem
>>>>>>>
>>>>>>> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
>>>>>>> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
>>>>>>> update t_order set status=’start’ where usr_id=’tom’;
>>>>>>> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
>>>>>>> Insert into t_order values(?,?,?) ;
>>>>>>
>>>>>> I think we can break the whole SQL interactions into smaller steps,
>>>>>> and let saga-actuator do the job one by one.
>>>>>> In this case, you can break the whole calling graph into several sub
>>>>>> calling graphs.
>>>>>> Any thought?
>>>>>>
>>>>>>> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
>>>>>>>
>>>>>>> so we should integrate saga-transaction like omega send event log step by step.
>>>>>>> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
>>>>>>> I feel that embed saga should have following capability.
>>>>>>> 1).It can provide service in jar package independent
>>>>>>> 2).each embed saga only recovery their own transaction-data of this instance.
>>>>>>> If one instance crashed, we can introduce external service to do failover.
>>>>>>>
>>>>>>> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
>>>>>>> of course, there have many things we should do.
>>>>>>>
>>>>>>> ------------------
>>>>>>> Zhao Jun
>>>>>>> Apache Sharding-Sphere & ServiceComb
>>>>>>>
>>>>>>>> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> First Saga actuator need to build up the calling grapha before sending
>>>>>>>> out the request. I don't think you can do the step by step SQL
>>>>>>>> invocation with Saga actuator.
>>>>>>>> If you want to call the SQL execution step by step , you may need to
>>>>>>>> switch to ServiceComb Pack project which has a coordinator to take
>>>>>>>> care of the distributed transaction. But that introduce another
>>>>>>>> endpoint(Alpha) to shardingsphere.
>>>>>>>>
>>>>>>>> From my understanding, Saga actuator is most efficient way to execute
>>>>>>>> the SQL across different data nodes.
>>>>>>>>
>>>>>>>>
>>>>>>>> Willem Jiang
>>>>>>>>
>>>>>>>> Twitter: willemjiang
>>>>>>>> Weibo: 姜宁willem
>>>>>>>>
>>>>>>>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
>>>>>>>>>
>>>>>>>>> Hi, all
>>>>>>>>>
>>>>>>>>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
>>>>>>>>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
>>>>>>>>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
>>>>>>>>> so recovery thread will not be executed correctly.
>>>>>>>>>
>>>>>>>>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
>>>>>>>>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
>>>>>>>>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
>>>>>>>>>
>>>>>>>>> any thought?
>>>>>>>>>
>>>>>>>>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------
>>>>>>>>> Zhao Jun
>>>>>>>>> Apache Sharding-Sphere & ServiceComb
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by zhaojun <zh...@126.com>.
I have just created a issue[1] in JIRA
[1]: https://issues.apache.org/jira/browse/SCB-1276 <https://issues.apache.org/jira/browse/SCB-1276>
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
> On May 5, 2019, at 8:46 AM, zhaojun <zh...@126.com> wrote:
>
> OK, I’ll try to write a JIRA today.
>
> ------------------
> Zhao Jun
> Apache Sharding-Sphere & ServiceComb
>
>
>> On May 2, 2019, at 3:18 PM, Willem Jiang <wi...@gmail.com> wrote:
>>
>> It sounds good to me.
>>
>> Willem Jiang
>>
>> Twitter: willemjiang
>> Weibo: 姜宁willem
>>
>> On Thu, May 2, 2019 at 11:14 AM 赵俊 <zh...@jd.com> wrote:
>>>
>>> Let SQL executor running command one by one is good to me.
>>>
>>> One logic-sql will be routed to multi-actual SQLs, then executed in sharding-sphere execute-engine.
>>> A actual SQL will be wrapped as a SQL execute-callback in sharding-sphere.
>>> We could provide a SPI for this SQL execute-callback.
>>> For integrated with saga, we could implement a saga execute-callback, it will wrap event sourcing logic like saga-actuator task have done.
>>>
>>>
>>>> On May 1, 2019, at 5:06 PM, Willem Jiang <wi...@gmail.com> wrote:
>>>>
>>>> We could do some enhancement on it.
>>>> From my understanding, We could create a calling graph which has three
>>>> sub graph of it, or we could let the SQL executor running the command
>>>> one by one.
>>>> Any thoughts?
>>>>
>>>> Willem Jiang
>>>>
>>>> Twitter: willemjiang
>>>> Weibo: 姜宁willem
>>>>
>>>> On Wed, May 1, 2019 at 4:29 PM zhaojun <zh...@126.com> wrote:
>>>>>
>>>>> But saga-actuator don’t support 3 logic SQL is a global transaction while you submit 3 times separately.
>>>>>
>>>>> ------------------
>>>>> Zhao Jun
>>>>> Apache Sharding-Sphere & ServiceComb
>>>>>
>>>>>> On May 1, 2019, at 8:38 AM, Willem Jiang <wi...@gmail.com> wrote:
>>>>>>
>>>>>> Willem Jiang
>>>>>>
>>>>>> Twitter: willemjiang
>>>>>> Weibo: 姜宁willem
>>>>>>
>>>>>> On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
>>>>>>>
>>>>>>> Hi, Willem
>>>>>>>
>>>>>>> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
>>>>>>> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
>>>>>>> update t_order set status=’start’ where usr_id=’tom’;
>>>>>>> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
>>>>>>> Insert into t_order values(?,?,?) ;
>>>>>>
>>>>>> I think we can break the whole SQL interactions into smaller steps,
>>>>>> and let saga-actuator do the job one by one.
>>>>>> In this case, you can break the whole calling graph into several sub
>>>>>> calling graphs.
>>>>>> Any thought?
>>>>>>
>>>>>>> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
>>>>>>>
>>>>>>> so we should integrate saga-transaction like omega send event log step by step.
>>>>>>> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
>>>>>>> I feel that embed saga should have following capability.
>>>>>>> 1).It can provide service in jar package independent
>>>>>>> 2).each embed saga only recovery their own transaction-data of this instance.
>>>>>>> If one instance crashed, we can introduce external service to do failover.
>>>>>>>
>>>>>>> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
>>>>>>> of course, there have many things we should do.
>>>>>>>
>>>>>>> ------------------
>>>>>>> Zhao Jun
>>>>>>> Apache Sharding-Sphere & ServiceComb
>>>>>>>
>>>>>>>> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> First Saga actuator need to build up the calling grapha before sending
>>>>>>>> out the request. I don't think you can do the step by step SQL
>>>>>>>> invocation with Saga actuator.
>>>>>>>> If you want to call the SQL execution step by step , you may need to
>>>>>>>> switch to ServiceComb Pack project which has a coordinator to take
>>>>>>>> care of the distributed transaction. But that introduce another
>>>>>>>> endpoint(Alpha) to shardingsphere.
>>>>>>>>
>>>>>>>> From my understanding, Saga actuator is most efficient way to execute
>>>>>>>> the SQL across different data nodes.
>>>>>>>>
>>>>>>>>
>>>>>>>> Willem Jiang
>>>>>>>>
>>>>>>>> Twitter: willemjiang
>>>>>>>> Weibo: 姜宁willem
>>>>>>>>
>>>>>>>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
>>>>>>>>>
>>>>>>>>> Hi, all
>>>>>>>>>
>>>>>>>>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
>>>>>>>>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
>>>>>>>>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
>>>>>>>>> so recovery thread will not be executed correctly.
>>>>>>>>>
>>>>>>>>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
>>>>>>>>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
>>>>>>>>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
>>>>>>>>>
>>>>>>>>> any thought?
>>>>>>>>>
>>>>>>>>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------
>>>>>>>>> Zhao Jun
>>>>>>>>> Apache Sharding-Sphere & ServiceComb
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by zhaojun <zh...@126.com>.
OK, I’ll try to write a JIRA today.
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
> On May 2, 2019, at 3:18 PM, Willem Jiang <wi...@gmail.com> wrote:
>
> It sounds good to me.
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Thu, May 2, 2019 at 11:14 AM 赵俊 <zh...@jd.com> wrote:
>>
>> Let SQL executor running command one by one is good to me.
>>
>> One logic-sql will be routed to multi-actual SQLs, then executed in sharding-sphere execute-engine.
>> A actual SQL will be wrapped as a SQL execute-callback in sharding-sphere.
>> We could provide a SPI for this SQL execute-callback.
>> For integrated with saga, we could implement a saga execute-callback, it will wrap event sourcing logic like saga-actuator task have done.
>>
>>
>>> On May 1, 2019, at 5:06 PM, Willem Jiang <wi...@gmail.com> wrote:
>>>
>>> We could do some enhancement on it.
>>> From my understanding, We could create a calling graph which has three
>>> sub graph of it, or we could let the SQL executor running the command
>>> one by one.
>>> Any thoughts?
>>>
>>> Willem Jiang
>>>
>>> Twitter: willemjiang
>>> Weibo: 姜宁willem
>>>
>>> On Wed, May 1, 2019 at 4:29 PM zhaojun <zh...@126.com> wrote:
>>>>
>>>> But saga-actuator don’t support 3 logic SQL is a global transaction while you submit 3 times separately.
>>>>
>>>> ------------------
>>>> Zhao Jun
>>>> Apache Sharding-Sphere & ServiceComb
>>>>
>>>>> On May 1, 2019, at 8:38 AM, Willem Jiang <wi...@gmail.com> wrote:
>>>>>
>>>>> Willem Jiang
>>>>>
>>>>> Twitter: willemjiang
>>>>> Weibo: 姜宁willem
>>>>>
>>>>> On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
>>>>>>
>>>>>> Hi, Willem
>>>>>>
>>>>>> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
>>>>>> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
>>>>>> update t_order set status=’start’ where usr_id=’tom’;
>>>>>> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
>>>>>> Insert into t_order values(?,?,?) ;
>>>>>
>>>>> I think we can break the whole SQL interactions into smaller steps,
>>>>> and let saga-actuator do the job one by one.
>>>>> In this case, you can break the whole calling graph into several sub
>>>>> calling graphs.
>>>>> Any thought?
>>>>>
>>>>>> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
>>>>>>
>>>>>> so we should integrate saga-transaction like omega send event log step by step.
>>>>>> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
>>>>>> I feel that embed saga should have following capability.
>>>>>> 1).It can provide service in jar package independent
>>>>>> 2).each embed saga only recovery their own transaction-data of this instance.
>>>>>> If one instance crashed, we can introduce external service to do failover.
>>>>>>
>>>>>> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
>>>>>> of course, there have many things we should do.
>>>>>>
>>>>>> ------------------
>>>>>> Zhao Jun
>>>>>> Apache Sharding-Sphere & ServiceComb
>>>>>>
>>>>>>> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
>>>>>>>
>>>>>>> First Saga actuator need to build up the calling grapha before sending
>>>>>>> out the request. I don't think you can do the step by step SQL
>>>>>>> invocation with Saga actuator.
>>>>>>> If you want to call the SQL execution step by step , you may need to
>>>>>>> switch to ServiceComb Pack project which has a coordinator to take
>>>>>>> care of the distributed transaction. But that introduce another
>>>>>>> endpoint(Alpha) to shardingsphere.
>>>>>>>
>>>>>>> From my understanding, Saga actuator is most efficient way to execute
>>>>>>> the SQL across different data nodes.
>>>>>>>
>>>>>>>
>>>>>>> Willem Jiang
>>>>>>>
>>>>>>> Twitter: willemjiang
>>>>>>> Weibo: 姜宁willem
>>>>>>>
>>>>>>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
>>>>>>>>
>>>>>>>> Hi, all
>>>>>>>>
>>>>>>>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
>>>>>>>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
>>>>>>>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
>>>>>>>> so recovery thread will not be executed correctly.
>>>>>>>>
>>>>>>>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
>>>>>>>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
>>>>>>>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
>>>>>>>>
>>>>>>>> any thought?
>>>>>>>>
>>>>>>>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------
>>>>>>>> Zhao Jun
>>>>>>>> Apache Sharding-Sphere & ServiceComb
>>>>>>>>
>>>>>>
>>>>
>>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by zhaojun <zh...@126.com>.
OK, I’ll try to write a JIRA today.
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
> On May 2, 2019, at 3:18 PM, Willem Jiang <wi...@gmail.com> wrote:
>
> It sounds good to me.
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Thu, May 2, 2019 at 11:14 AM 赵俊 <zh...@jd.com> wrote:
>>
>> Let SQL executor running command one by one is good to me.
>>
>> One logic-sql will be routed to multi-actual SQLs, then executed in sharding-sphere execute-engine.
>> A actual SQL will be wrapped as a SQL execute-callback in sharding-sphere.
>> We could provide a SPI for this SQL execute-callback.
>> For integrated with saga, we could implement a saga execute-callback, it will wrap event sourcing logic like saga-actuator task have done.
>>
>>
>>> On May 1, 2019, at 5:06 PM, Willem Jiang <wi...@gmail.com> wrote:
>>>
>>> We could do some enhancement on it.
>>> From my understanding, We could create a calling graph which has three
>>> sub graph of it, or we could let the SQL executor running the command
>>> one by one.
>>> Any thoughts?
>>>
>>> Willem Jiang
>>>
>>> Twitter: willemjiang
>>> Weibo: 姜宁willem
>>>
>>> On Wed, May 1, 2019 at 4:29 PM zhaojun <zh...@126.com> wrote:
>>>>
>>>> But saga-actuator don’t support 3 logic SQL is a global transaction while you submit 3 times separately.
>>>>
>>>> ------------------
>>>> Zhao Jun
>>>> Apache Sharding-Sphere & ServiceComb
>>>>
>>>>> On May 1, 2019, at 8:38 AM, Willem Jiang <wi...@gmail.com> wrote:
>>>>>
>>>>> Willem Jiang
>>>>>
>>>>> Twitter: willemjiang
>>>>> Weibo: 姜宁willem
>>>>>
>>>>> On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
>>>>>>
>>>>>> Hi, Willem
>>>>>>
>>>>>> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
>>>>>> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
>>>>>> update t_order set status=’start’ where usr_id=’tom’;
>>>>>> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
>>>>>> Insert into t_order values(?,?,?) ;
>>>>>
>>>>> I think we can break the whole SQL interactions into smaller steps,
>>>>> and let saga-actuator do the job one by one.
>>>>> In this case, you can break the whole calling graph into several sub
>>>>> calling graphs.
>>>>> Any thought?
>>>>>
>>>>>> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
>>>>>>
>>>>>> so we should integrate saga-transaction like omega send event log step by step.
>>>>>> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
>>>>>> I feel that embed saga should have following capability.
>>>>>> 1).It can provide service in jar package independent
>>>>>> 2).each embed saga only recovery their own transaction-data of this instance.
>>>>>> If one instance crashed, we can introduce external service to do failover.
>>>>>>
>>>>>> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
>>>>>> of course, there have many things we should do.
>>>>>>
>>>>>> ------------------
>>>>>> Zhao Jun
>>>>>> Apache Sharding-Sphere & ServiceComb
>>>>>>
>>>>>>> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
>>>>>>>
>>>>>>> First Saga actuator need to build up the calling grapha before sending
>>>>>>> out the request. I don't think you can do the step by step SQL
>>>>>>> invocation with Saga actuator.
>>>>>>> If you want to call the SQL execution step by step , you may need to
>>>>>>> switch to ServiceComb Pack project which has a coordinator to take
>>>>>>> care of the distributed transaction. But that introduce another
>>>>>>> endpoint(Alpha) to shardingsphere.
>>>>>>>
>>>>>>> From my understanding, Saga actuator is most efficient way to execute
>>>>>>> the SQL across different data nodes.
>>>>>>>
>>>>>>>
>>>>>>> Willem Jiang
>>>>>>>
>>>>>>> Twitter: willemjiang
>>>>>>> Weibo: 姜宁willem
>>>>>>>
>>>>>>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
>>>>>>>>
>>>>>>>> Hi, all
>>>>>>>>
>>>>>>>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
>>>>>>>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
>>>>>>>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
>>>>>>>> so recovery thread will not be executed correctly.
>>>>>>>>
>>>>>>>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
>>>>>>>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
>>>>>>>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
>>>>>>>>
>>>>>>>> any thought?
>>>>>>>>
>>>>>>>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------
>>>>>>>> Zhao Jun
>>>>>>>> Apache Sharding-Sphere & ServiceComb
>>>>>>>>
>>>>>>
>>>>
>>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Willem Jiang <wi...@gmail.com>.
It sounds good to me.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Thu, May 2, 2019 at 11:14 AM 赵俊 <zh...@jd.com> wrote:
>
> Let SQL executor running command one by one is good to me.
>
> One logic-sql will be routed to multi-actual SQLs, then executed in sharding-sphere execute-engine.
> A actual SQL will be wrapped as a SQL execute-callback in sharding-sphere.
> We could provide a SPI for this SQL execute-callback.
> For integrated with saga, we could implement a saga execute-callback, it will wrap event sourcing logic like saga-actuator task have done.
>
>
> > On May 1, 2019, at 5:06 PM, Willem Jiang <wi...@gmail.com> wrote:
> >
> > We could do some enhancement on it.
> > From my understanding, We could create a calling graph which has three
> > sub graph of it, or we could let the SQL executor running the command
> > one by one.
> > Any thoughts?
> >
> > Willem Jiang
> >
> > Twitter: willemjiang
> > Weibo: 姜宁willem
> >
> > On Wed, May 1, 2019 at 4:29 PM zhaojun <zh...@126.com> wrote:
> >>
> >> But saga-actuator don’t support 3 logic SQL is a global transaction while you submit 3 times separately.
> >>
> >> ------------------
> >> Zhao Jun
> >> Apache Sharding-Sphere & ServiceComb
> >>
> >>> On May 1, 2019, at 8:38 AM, Willem Jiang <wi...@gmail.com> wrote:
> >>>
> >>> Willem Jiang
> >>>
> >>> Twitter: willemjiang
> >>> Weibo: 姜宁willem
> >>>
> >>> On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
> >>>>
> >>>> Hi, Willem
> >>>>
> >>>> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
> >>>> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
> >>>> update t_order set status=’start’ where usr_id=’tom’;
> >>>> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
> >>>> Insert into t_order values(?,?,?) ;
> >>>
> >>> I think we can break the whole SQL interactions into smaller steps,
> >>> and let saga-actuator do the job one by one.
> >>> In this case, you can break the whole calling graph into several sub
> >>> calling graphs.
> >>> Any thought?
> >>>
> >>>> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
> >>>>
> >>>> so we should integrate saga-transaction like omega send event log step by step.
> >>>> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
> >>>> I feel that embed saga should have following capability.
> >>>> 1).It can provide service in jar package independent
> >>>> 2).each embed saga only recovery their own transaction-data of this instance.
> >>>> If one instance crashed, we can introduce external service to do failover.
> >>>>
> >>>> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
> >>>> of course, there have many things we should do.
> >>>>
> >>>> ------------------
> >>>> Zhao Jun
> >>>> Apache Sharding-Sphere & ServiceComb
> >>>>
> >>>>> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
> >>>>>
> >>>>> First Saga actuator need to build up the calling grapha before sending
> >>>>> out the request. I don't think you can do the step by step SQL
> >>>>> invocation with Saga actuator.
> >>>>> If you want to call the SQL execution step by step , you may need to
> >>>>> switch to ServiceComb Pack project which has a coordinator to take
> >>>>> care of the distributed transaction. But that introduce another
> >>>>> endpoint(Alpha) to shardingsphere.
> >>>>>
> >>>>> From my understanding, Saga actuator is most efficient way to execute
> >>>>> the SQL across different data nodes.
> >>>>>
> >>>>>
> >>>>> Willem Jiang
> >>>>>
> >>>>> Twitter: willemjiang
> >>>>> Weibo: 姜宁willem
> >>>>>
> >>>>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
> >>>>>>
> >>>>>> Hi, all
> >>>>>>
> >>>>>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
> >>>>>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
> >>>>>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
> >>>>>> so recovery thread will not be executed correctly.
> >>>>>>
> >>>>>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
> >>>>>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
> >>>>>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
> >>>>>>
> >>>>>> any thought?
> >>>>>>
> >>>>>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> ------------------
> >>>>>> Zhao Jun
> >>>>>> Apache Sharding-Sphere & ServiceComb
> >>>>>>
> >>>>
> >>
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Willem Jiang <wi...@gmail.com>.
It sounds good to me.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Thu, May 2, 2019 at 11:14 AM 赵俊 <zh...@jd.com> wrote:
>
> Let SQL executor running command one by one is good to me.
>
> One logic-sql will be routed to multi-actual SQLs, then executed in sharding-sphere execute-engine.
> A actual SQL will be wrapped as a SQL execute-callback in sharding-sphere.
> We could provide a SPI for this SQL execute-callback.
> For integrated with saga, we could implement a saga execute-callback, it will wrap event sourcing logic like saga-actuator task have done.
>
>
> > On May 1, 2019, at 5:06 PM, Willem Jiang <wi...@gmail.com> wrote:
> >
> > We could do some enhancement on it.
> > From my understanding, We could create a calling graph which has three
> > sub graph of it, or we could let the SQL executor running the command
> > one by one.
> > Any thoughts?
> >
> > Willem Jiang
> >
> > Twitter: willemjiang
> > Weibo: 姜宁willem
> >
> > On Wed, May 1, 2019 at 4:29 PM zhaojun <zh...@126.com> wrote:
> >>
> >> But saga-actuator don’t support 3 logic SQL is a global transaction while you submit 3 times separately.
> >>
> >> ------------------
> >> Zhao Jun
> >> Apache Sharding-Sphere & ServiceComb
> >>
> >>> On May 1, 2019, at 8:38 AM, Willem Jiang <wi...@gmail.com> wrote:
> >>>
> >>> Willem Jiang
> >>>
> >>> Twitter: willemjiang
> >>> Weibo: 姜宁willem
> >>>
> >>> On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
> >>>>
> >>>> Hi, Willem
> >>>>
> >>>> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
> >>>> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
> >>>> update t_order set status=’start’ where usr_id=’tom’;
> >>>> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
> >>>> Insert into t_order values(?,?,?) ;
> >>>
> >>> I think we can break the whole SQL interactions into smaller steps,
> >>> and let saga-actuator do the job one by one.
> >>> In this case, you can break the whole calling graph into several sub
> >>> calling graphs.
> >>> Any thought?
> >>>
> >>>> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
> >>>>
> >>>> so we should integrate saga-transaction like omega send event log step by step.
> >>>> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
> >>>> I feel that embed saga should have following capability.
> >>>> 1).It can provide service in jar package independent
> >>>> 2).each embed saga only recovery their own transaction-data of this instance.
> >>>> If one instance crashed, we can introduce external service to do failover.
> >>>>
> >>>> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
> >>>> of course, there have many things we should do.
> >>>>
> >>>> ------------------
> >>>> Zhao Jun
> >>>> Apache Sharding-Sphere & ServiceComb
> >>>>
> >>>>> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
> >>>>>
> >>>>> First Saga actuator need to build up the calling grapha before sending
> >>>>> out the request. I don't think you can do the step by step SQL
> >>>>> invocation with Saga actuator.
> >>>>> If you want to call the SQL execution step by step , you may need to
> >>>>> switch to ServiceComb Pack project which has a coordinator to take
> >>>>> care of the distributed transaction. But that introduce another
> >>>>> endpoint(Alpha) to shardingsphere.
> >>>>>
> >>>>> From my understanding, Saga actuator is most efficient way to execute
> >>>>> the SQL across different data nodes.
> >>>>>
> >>>>>
> >>>>> Willem Jiang
> >>>>>
> >>>>> Twitter: willemjiang
> >>>>> Weibo: 姜宁willem
> >>>>>
> >>>>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
> >>>>>>
> >>>>>> Hi, all
> >>>>>>
> >>>>>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
> >>>>>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
> >>>>>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
> >>>>>> so recovery thread will not be executed correctly.
> >>>>>>
> >>>>>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
> >>>>>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
> >>>>>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
> >>>>>>
> >>>>>> any thought?
> >>>>>>
> >>>>>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> ------------------
> >>>>>> Zhao Jun
> >>>>>> Apache Sharding-Sphere & ServiceComb
> >>>>>>
> >>>>
> >>
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by 赵俊 <zh...@jd.com>.
Let SQL executor running command one by one is good to me.
One logic-sql will be routed to multi-actual SQLs, then executed in sharding-sphere execute-engine.
A actual SQL will be wrapped as a SQL execute-callback in sharding-sphere.
We could provide a SPI for this SQL execute-callback.
For integrated with saga, we could implement a saga execute-callback, it will wrap event sourcing logic like saga-actuator task have done.
> On May 1, 2019, at 5:06 PM, Willem Jiang <wi...@gmail.com> wrote:
>
> We could do some enhancement on it.
> From my understanding, We could create a calling graph which has three
> sub graph of it, or we could let the SQL executor running the command
> one by one.
> Any thoughts?
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Wed, May 1, 2019 at 4:29 PM zhaojun <zh...@126.com> wrote:
>>
>> But saga-actuator don’t support 3 logic SQL is a global transaction while you submit 3 times separately.
>>
>> ------------------
>> Zhao Jun
>> Apache Sharding-Sphere & ServiceComb
>>
>>> On May 1, 2019, at 8:38 AM, Willem Jiang <wi...@gmail.com> wrote:
>>>
>>> Willem Jiang
>>>
>>> Twitter: willemjiang
>>> Weibo: 姜宁willem
>>>
>>> On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
>>>>
>>>> Hi, Willem
>>>>
>>>> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
>>>> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
>>>> update t_order set status=’start’ where usr_id=’tom’;
>>>> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
>>>> Insert into t_order values(?,?,?) ;
>>>
>>> I think we can break the whole SQL interactions into smaller steps,
>>> and let saga-actuator do the job one by one.
>>> In this case, you can break the whole calling graph into several sub
>>> calling graphs.
>>> Any thought?
>>>
>>>> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
>>>>
>>>> so we should integrate saga-transaction like omega send event log step by step.
>>>> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
>>>> I feel that embed saga should have following capability.
>>>> 1).It can provide service in jar package independent
>>>> 2).each embed saga only recovery their own transaction-data of this instance.
>>>> If one instance crashed, we can introduce external service to do failover.
>>>>
>>>> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
>>>> of course, there have many things we should do.
>>>>
>>>> ------------------
>>>> Zhao Jun
>>>> Apache Sharding-Sphere & ServiceComb
>>>>
>>>>> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
>>>>>
>>>>> First Saga actuator need to build up the calling grapha before sending
>>>>> out the request. I don't think you can do the step by step SQL
>>>>> invocation with Saga actuator.
>>>>> If you want to call the SQL execution step by step , you may need to
>>>>> switch to ServiceComb Pack project which has a coordinator to take
>>>>> care of the distributed transaction. But that introduce another
>>>>> endpoint(Alpha) to shardingsphere.
>>>>>
>>>>> From my understanding, Saga actuator is most efficient way to execute
>>>>> the SQL across different data nodes.
>>>>>
>>>>>
>>>>> Willem Jiang
>>>>>
>>>>> Twitter: willemjiang
>>>>> Weibo: 姜宁willem
>>>>>
>>>>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
>>>>>>
>>>>>> Hi, all
>>>>>>
>>>>>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
>>>>>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
>>>>>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
>>>>>> so recovery thread will not be executed correctly.
>>>>>>
>>>>>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
>>>>>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
>>>>>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
>>>>>>
>>>>>> any thought?
>>>>>>
>>>>>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------
>>>>>> Zhao Jun
>>>>>> Apache Sharding-Sphere & ServiceComb
>>>>>>
>>>>
>>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Willem Jiang <wi...@gmail.com>.
We could do some enhancement on it.
From my understanding, We could create a calling graph which has three
sub graph of it, or we could let the SQL executor running the command
one by one.
Any thoughts?
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Wed, May 1, 2019 at 4:29 PM zhaojun <zh...@126.com> wrote:
>
> But saga-actuator don’t support 3 logic SQL is a global transaction while you submit 3 times separately.
>
> ------------------
> Zhao Jun
> Apache Sharding-Sphere & ServiceComb
>
> > On May 1, 2019, at 8:38 AM, Willem Jiang <wi...@gmail.com> wrote:
> >
> > Willem Jiang
> >
> > Twitter: willemjiang
> > Weibo: 姜宁willem
> >
> > On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
> >>
> >> Hi, Willem
> >>
> >> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
> >> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
> >> update t_order set status=’start’ where usr_id=’tom’;
> >> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
> >> Insert into t_order values(?,?,?) ;
> >
> > I think we can break the whole SQL interactions into smaller steps,
> > and let saga-actuator do the job one by one.
> > In this case, you can break the whole calling graph into several sub
> > calling graphs.
> > Any thought?
> >
> >> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
> >>
> >> so we should integrate saga-transaction like omega send event log step by step.
> >> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
> >> I feel that embed saga should have following capability.
> >> 1).It can provide service in jar package independent
> >> 2).each embed saga only recovery their own transaction-data of this instance.
> >> If one instance crashed, we can introduce external service to do failover.
> >>
> >> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
> >> of course, there have many things we should do.
> >>
> >> ------------------
> >> Zhao Jun
> >> Apache Sharding-Sphere & ServiceComb
> >>
> >>> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
> >>>
> >>> First Saga actuator need to build up the calling grapha before sending
> >>> out the request. I don't think you can do the step by step SQL
> >>> invocation with Saga actuator.
> >>> If you want to call the SQL execution step by step , you may need to
> >>> switch to ServiceComb Pack project which has a coordinator to take
> >>> care of the distributed transaction. But that introduce another
> >>> endpoint(Alpha) to shardingsphere.
> >>>
> >>> From my understanding, Saga actuator is most efficient way to execute
> >>> the SQL across different data nodes.
> >>>
> >>>
> >>> Willem Jiang
> >>>
> >>> Twitter: willemjiang
> >>> Weibo: 姜宁willem
> >>>
> >>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
> >>>>
> >>>> Hi, all
> >>>>
> >>>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
> >>>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
> >>>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
> >>>> so recovery thread will not be executed correctly.
> >>>>
> >>>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
> >>>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
> >>>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
> >>>>
> >>>> any thought?
> >>>>
> >>>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
> >>>>
> >>>>
> >>>>
> >>>> ------------------
> >>>> Zhao Jun
> >>>> Apache Sharding-Sphere & ServiceComb
> >>>>
> >>
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by zhaojun <zh...@126.com>.
But saga-actuator don’t support 3 logic SQL is a global transaction while you submit 3 times separately.
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
> On May 1, 2019, at 8:38 AM, Willem Jiang <wi...@gmail.com> wrote:
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
>>
>> Hi, Willem
>>
>> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
>> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
>> update t_order set status=’start’ where usr_id=’tom’;
>> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
>> Insert into t_order values(?,?,?) ;
>
> I think we can break the whole SQL interactions into smaller steps,
> and let saga-actuator do the job one by one.
> In this case, you can break the whole calling graph into several sub
> calling graphs.
> Any thought?
>
>> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
>>
>> so we should integrate saga-transaction like omega send event log step by step.
>> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
>> I feel that embed saga should have following capability.
>> 1).It can provide service in jar package independent
>> 2).each embed saga only recovery their own transaction-data of this instance.
>> If one instance crashed, we can introduce external service to do failover.
>>
>> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
>> of course, there have many things we should do.
>>
>> ------------------
>> Zhao Jun
>> Apache Sharding-Sphere & ServiceComb
>>
>>> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
>>>
>>> First Saga actuator need to build up the calling grapha before sending
>>> out the request. I don't think you can do the step by step SQL
>>> invocation with Saga actuator.
>>> If you want to call the SQL execution step by step , you may need to
>>> switch to ServiceComb Pack project which has a coordinator to take
>>> care of the distributed transaction. But that introduce another
>>> endpoint(Alpha) to shardingsphere.
>>>
>>> From my understanding, Saga actuator is most efficient way to execute
>>> the SQL across different data nodes.
>>>
>>>
>>> Willem Jiang
>>>
>>> Twitter: willemjiang
>>> Weibo: 姜宁willem
>>>
>>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
>>>>
>>>> Hi, all
>>>>
>>>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
>>>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
>>>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
>>>> so recovery thread will not be executed correctly.
>>>>
>>>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
>>>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
>>>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
>>>>
>>>> any thought?
>>>>
>>>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
>>>>
>>>>
>>>>
>>>> ------------------
>>>> Zhao Jun
>>>> Apache Sharding-Sphere & ServiceComb
>>>>
>>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by zhaojun <zh...@126.com>.
But saga-actuator don’t support 3 logic SQL is a global transaction while you submit 3 times separately.
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
> On May 1, 2019, at 8:38 AM, Willem Jiang <wi...@gmail.com> wrote:
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
>>
>> Hi, Willem
>>
>> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
>> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
>> update t_order set status=’start’ where usr_id=’tom’;
>> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
>> Insert into t_order values(?,?,?) ;
>
> I think we can break the whole SQL interactions into smaller steps,
> and let saga-actuator do the job one by one.
> In this case, you can break the whole calling graph into several sub
> calling graphs.
> Any thought?
>
>> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
>>
>> so we should integrate saga-transaction like omega send event log step by step.
>> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
>> I feel that embed saga should have following capability.
>> 1).It can provide service in jar package independent
>> 2).each embed saga only recovery their own transaction-data of this instance.
>> If one instance crashed, we can introduce external service to do failover.
>>
>> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
>> of course, there have many things we should do.
>>
>> ------------------
>> Zhao Jun
>> Apache Sharding-Sphere & ServiceComb
>>
>>> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
>>>
>>> First Saga actuator need to build up the calling grapha before sending
>>> out the request. I don't think you can do the step by step SQL
>>> invocation with Saga actuator.
>>> If you want to call the SQL execution step by step , you may need to
>>> switch to ServiceComb Pack project which has a coordinator to take
>>> care of the distributed transaction. But that introduce another
>>> endpoint(Alpha) to shardingsphere.
>>>
>>> From my understanding, Saga actuator is most efficient way to execute
>>> the SQL across different data nodes.
>>>
>>>
>>> Willem Jiang
>>>
>>> Twitter: willemjiang
>>> Weibo: 姜宁willem
>>>
>>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
>>>>
>>>> Hi, all
>>>>
>>>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
>>>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
>>>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
>>>> so recovery thread will not be executed correctly.
>>>>
>>>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
>>>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
>>>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
>>>>
>>>> any thought?
>>>>
>>>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
>>>>
>>>>
>>>>
>>>> ------------------
>>>> Zhao Jun
>>>> Apache Sharding-Sphere & ServiceComb
>>>>
>>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Willem Jiang <wi...@gmail.com>.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
>
> Hi, Willem
>
> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
> update t_order set status=’start’ where usr_id=’tom’;
> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
> Insert into t_order values(?,?,?) ;
I think we can break the whole SQL interactions into smaller steps,
and let saga-actuator do the job one by one.
In this case, you can break the whole calling graph into several sub
calling graphs.
Any thought?
> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
>
> so we should integrate saga-transaction like omega send event log step by step.
> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
> I feel that embed saga should have following capability.
> 1).It can provide service in jar package independent
> 2).each embed saga only recovery their own transaction-data of this instance.
> If one instance crashed, we can introduce external service to do failover.
>
> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
> of course, there have many things we should do.
>
> ------------------
> Zhao Jun
> Apache Sharding-Sphere & ServiceComb
>
> > On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
> >
> > First Saga actuator need to build up the calling grapha before sending
> > out the request. I don't think you can do the step by step SQL
> > invocation with Saga actuator.
> > If you want to call the SQL execution step by step , you may need to
> > switch to ServiceComb Pack project which has a coordinator to take
> > care of the distributed transaction. But that introduce another
> > endpoint(Alpha) to shardingsphere.
> >
> > From my understanding, Saga actuator is most efficient way to execute
> > the SQL across different data nodes.
> >
> >
> > Willem Jiang
> >
> > Twitter: willemjiang
> > Weibo: 姜宁willem
> >
> > On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
> >>
> >> Hi, all
> >>
> >> currently, we have integrated with saga using graph based engine in shardingsphere[1]
> >> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
> >> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
> >> so recovery thread will not be executed correctly.
> >>
> >> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
> >> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
> >> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
> >>
> >> any thought?
> >>
> >> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
> >>
> >>
> >>
> >> ------------------
> >> Zhao Jun
> >> Apache Sharding-Sphere & ServiceComb
> >>
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Willem Jiang <wi...@gmail.com>.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Mon, Apr 29, 2019 at 7:58 PM zhaojun <zh...@126.com> wrote:
>
> Hi, Willem
>
> Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
> 1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
> update t_order set status=’start’ where usr_id=’tom’;
> select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
> Insert into t_order values(?,?,?) ;
I think we can break the whole SQL interactions into smaller steps,
and let saga-actuator do the job one by one.
In this case, you can break the whole calling graph into several sub
calling graphs.
Any thought?
> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
>
> so we should integrate saga-transaction like omega send event log step by step.
> It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
> I feel that embed saga should have following capability.
> 1).It can provide service in jar package independent
> 2).each embed saga only recovery their own transaction-data of this instance.
> If one instance crashed, we can introduce external service to do failover.
>
> so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
> of course, there have many things we should do.
>
> ------------------
> Zhao Jun
> Apache Sharding-Sphere & ServiceComb
>
> > On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
> >
> > First Saga actuator need to build up the calling grapha before sending
> > out the request. I don't think you can do the step by step SQL
> > invocation with Saga actuator.
> > If you want to call the SQL execution step by step , you may need to
> > switch to ServiceComb Pack project which has a coordinator to take
> > care of the distributed transaction. But that introduce another
> > endpoint(Alpha) to shardingsphere.
> >
> > From my understanding, Saga actuator is most efficient way to execute
> > the SQL across different data nodes.
> >
> >
> > Willem Jiang
> >
> > Twitter: willemjiang
> > Weibo: 姜宁willem
> >
> > On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
> >>
> >> Hi, all
> >>
> >> currently, we have integrated with saga using graph based engine in shardingsphere[1]
> >> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
> >> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
> >> so recovery thread will not be executed correctly.
> >>
> >> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
> >> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
> >> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
> >>
> >> any thought?
> >>
> >> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
> >>
> >>
> >>
> >> ------------------
> >> Zhao Jun
> >> Apache Sharding-Sphere & ServiceComb
> >>
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by zhaojun <zh...@126.com>.
Hi, Willem
Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
update t_order set status=’start’ where usr_id=’tom’;
select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
Insert into t_order values(?,?,?) ;
2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
so we should integrate saga-transaction like omega send event log step by step.
It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
I feel that embed saga should have following capability.
1).It can provide service in jar package independent
2).each embed saga only recovery their own transaction-data of this instance.
If one instance crashed, we can introduce external service to do failover.
so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
of course, there have many things we should do.
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
>
> First Saga actuator need to build up the calling grapha before sending
> out the request. I don't think you can do the step by step SQL
> invocation with Saga actuator.
> If you want to call the SQL execution step by step , you may need to
> switch to ServiceComb Pack project which has a coordinator to take
> care of the distributed transaction. But that introduce another
> endpoint(Alpha) to shardingsphere.
>
> From my understanding, Saga actuator is most efficient way to execute
> the SQL across different data nodes.
>
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
>>
>> Hi, all
>>
>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
>> so recovery thread will not be executed correctly.
>>
>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
>>
>> any thought?
>>
>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
>>
>>
>>
>> ------------------
>> Zhao Jun
>> Apache Sharding-Sphere & ServiceComb
>>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by zhaojun <zh...@126.com>.
Hi, Willem
Saga actuator could not compatible with ShardingSphere currently, there are 2 main problem exist.
1. If sql execute in saga transport instead of ShardingSphere, we can not see the result of logic-sql even in one transaction, it is like this:
update t_order set status=’start’ where usr_id=’tom’;
select status from t_order where usr_id=’tom’; —> we can’t query ’start’ record as actuator haven’t executed.
Insert into t_order values(?,?,?) ;
2. If logic-sql execute in ShardingSphere, we cannot handle recovery before submit graph result, as event log only wrote at saga engine.
so we should integrate saga-transaction like omega send event log step by step.
It is better to make every instance do recovery Independently, instead of providing another coordinator center service.
I feel that embed saga should have following capability.
1).It can provide service in jar package independent
2).each embed saga only recovery their own transaction-data of this instance.
If one instance crashed, we can introduce external service to do failover.
so we consider about exending saga-acutator, if it can support submit task step by step in one transaction, it is a good choice.
of course, there have many things we should do.
------------------
Zhao Jun
Apache Sharding-Sphere & ServiceComb
> On Apr 29, 2019, at 5:17 PM, Willem Jiang <wi...@gmail.com> wrote:
>
> First Saga actuator need to build up the calling grapha before sending
> out the request. I don't think you can do the step by step SQL
> invocation with Saga actuator.
> If you want to call the SQL execution step by step , you may need to
> switch to ServiceComb Pack project which has a coordinator to take
> care of the distributed transaction. But that introduce another
> endpoint(Alpha) to shardingsphere.
>
> From my understanding, Saga actuator is most efficient way to execute
> the SQL across different data nodes.
>
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
>>
>> Hi, all
>>
>> currently, we have integrated with saga using graph based engine in shardingsphere[1]
>> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
>> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
>> so recovery thread will not be executed correctly.
>>
>> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
>> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
>> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
>>
>> any thought?
>>
>> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
>>
>>
>>
>> ------------------
>> Zhao Jun
>> Apache Sharding-Sphere & ServiceComb
>>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Willem Jiang <wi...@gmail.com>.
First Saga actuator need to build up the calling grapha before sending
out the request. I don't think you can do the step by step SQL
invocation with Saga actuator.
If you want to call the SQL execution step by step , you may need to
switch to ServiceComb Pack project which has a coordinator to take
care of the distributed transaction. But that introduce another
endpoint(Alpha) to shardingsphere.
From my understanding, Saga actuator is most efficient way to execute
the SQL across different data nodes.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
>
> Hi, all
>
> currently, we have integrated with saga using graph based engine in shardingsphere[1]
> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
> so recovery thread will not be executed correctly.
>
> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
>
> any thought?
>
> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
>
>
>
> ------------------
> Zhao Jun
> Apache Sharding-Sphere & ServiceComb
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Willem Jiang <wi...@gmail.com>.
First Saga actuator need to build up the calling grapha before sending
out the request. I don't think you can do the step by step SQL
invocation with Saga actuator.
If you want to call the SQL execution step by step , you may need to
switch to ServiceComb Pack project which has a coordinator to take
care of the distributed transaction. But that introduce another
endpoint(Alpha) to shardingsphere.
From my understanding, Saga actuator is most efficient way to execute
the SQL across different data nodes.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Fri, Apr 26, 2019 at 6:41 PM zhaojun <zh...@126.com> wrote:
>
> Hi, all
>
> currently, we have integrated with saga using graph based engine in shardingsphere[1]
> it need us to collect all participated actual SQL, then submit to saga actuator in commit/rollback phase.
> if application crashed before invoking saga actuator, undo log of branch transaction SQL will not be saved,
> so recovery thread will not be executed correctly.
>
> it's better that encapsulating every actual SQL as a saga task in shardingsphere side,
> then submit to saga actuator realtime instead of batch processing all the SQLs at commit/rollback phase.
> this architecture will make the boundary more clear between shardingsphre and saga, currently we have done some additional works for integrating saga.
>
> any thought?
>
> [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga <https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga>
>
>
>
> ------------------
> Zhao Jun
> Apache Sharding-Sphere & ServiceComb
>
Re: [saga-actuator]consider about adding StreamBasedSaga engine to
integrate with shardingsphere
Posted by Zheng Feng <zh...@gmail.com>.
Hi Zhao Jun,
It looks good to me and it could have the revert SQL in the compensate
method automatically ?
Regards,
Zheng Feng
zhaojun <zh...@126.com> 于2019年4月26日周五 下午6:41写道：
> Hi, all
>
> currently, we have integrated with saga using graph based engine in
> shardingsphere[1]
> it need us to collect all participated actual SQL, then submit to saga
> actuator in commit/rollback phase.
> if application crashed before invoking saga actuator, undo log of branch
> transaction SQL will not be saved,
> so recovery thread will not be executed correctly.
>
> it's better that encapsulating every actual SQL as a saga task in
> shardingsphere side,
> then submit to saga actuator realtime instead of batch processing all the
> SQLs at commit/rollback phase.
> this architecture will make the boundary more clear between shardingsphre
> and saga, currently we have done some additional works for integrating saga.
>
> any thought?
>
> [1]:
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> <
> https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
> >
>
>
>
> ------------------
> Zhao Jun
> Apache Sharding-Sphere & ServiceComb
>
>