You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@skywalking.apache.org by Juan Pan <pa...@apache.org> on 2020/02/16 10:36:06 UTC
[DISCUSS] Add Apache ShardingSphere as the server storage for
Apache SkyWalking
Hi fellows,
I am working on providing ShardingSphere[1] as one of the server storages for Skywalking.
ShardingSphere will help shard the data collected by Skywalking into different databases(Popular RDBMS, like MySQL, PostgreSQL and so on) and tables to seek better query performance and do scale-out of database.
Since it is not an easy job, and needs some helps from community, i put task schedule and discussion items here to welcome your comments and suggestions.
If you are interested in this one, welcome to be my partner! :-)
------------
Here is the schedule,
1. Acquire all required tables and query SQLs from db logs, like general log of MySQL. [DONE]
2. Analyse those tables and query SQLs to explore appropriate sharding strategy.[DOING]
3. Provide class implementation of `PreciseShardingAlgorithm` and `RangeShardingAlgorithm` of ShardingSphere to do sharding.[TODO]
4. Provide configuration of Sharding Proxy[2] for Skywalking.[TODO]
5. Provide Acceptance test(maybe e2e tests of Skywalking?).[TODO]
6. Update readme[3].[TODO]
7. Update storage doc[4].[TODO]
-------------
It seems a long-term work, and i will do it step by step. Any progress will be updated here, also ISSUE [5] will give you thread of PRs and discussion detail.
[1] https://shardingsphere.apache.org/
[2] https://shardingsphere.apache.org/document/current/en/quick-start/sharding-proxy-quick-start/
[3] https://github.com/apache/skywalking/tree/master/install
[4 ]https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-storage.md
[5] https://github.com/apache/skywalking/issues/4131
Juan Pan (Trista)
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: panjuan@apache.org
Re: [DISCUSS] Add Apache ShardingSphere as the server storage for
Apache SkyWalking
Posted by Sheng Wu <wu...@gmail.com>.
Hi Juan Pan
I am updating the 8.0.0 core in the PR[1].
I removed the `-day`, `-month`, `-hour` tables. And *_inventory tables have
been removed, with several metadata tables.
Please recheck the storage, the config file is required to change.
[1] https://github.com/apache/skywalking/pull/4599
Sheng Wu 吴晟
Twitter, wusheng1108
Sheng Wu <wu...@gmail.com> 于2020年3月16日周一 上午8:57写道：
> Thinks happen quickly, I think that PR is going to be merged soon.
> You could wait 2-3 days.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
>
> Juan Pan <pa...@apache.org> 于2020年3月15日周日 下午3:39写道：
>
>> Hi,
>>
>>
>> Thanks for your kind remainder.
>> Actually i am on the way writing e2e tests for ShardingSphere.
>> Considering this PR[1] has not been merged and the discussion is underway,
>> do you think i should stop my current work (Based on the old one) and
>> wait for your progress?
>>
>>
>> Cheers,
>> Trista
>>
>>
>> Juan Pan (Trista)
>>
>> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
>> E-mail: panjuan@apache.org
>>
>>
>>
>>
>> On 03/15/2020 14:08，kezhenxu94<ke...@apache.org> wrote：
>> Hi Juan Pan, we've noticed that the E2E tests are now very unfriendly for
>> new contributors, thus we have rewrote the tests now and, from my point of
>> view, if you're familiar with Docker and docker compose, you should be
>> quick to get started with the new E2E framework, please take a look at this
>> pull request https://github.com/apache/skywalking/pull/4510 and check
>> out the branch locally to play around it, and I hope to get feedback from
>> E2E contributors like you whether it improves, compared with the old one,
>> thanks.
>>
>>
>>
>>
>>
>>
>> --
>>
>> GitHub @kezhenxu94
>> Apache SkyWalking, Apache Dubbo
>>
>>
>>
>> At 2020-03-15 13:42:55, "Juan Pan" <pa...@apache.org> wrote:
>> Hi community,
>>
>>
>> Update task status and raise my questions. Could I get your bits of help?
>>
>>
>> According to our schedule, the 1st to 3rd sub-tasks have been completed
>> already[1]. Currently, I am working on sub-task 4th and 5th, i.e.,
>> 4. Provide configuration of Sharding Proxy[2] for Skywalking.
>> 5. Provide e2e tests.
>>
>>
>> Here are my questions,
>> 1. `rc0-prepare.sh` is used only for one `image` of
>> `docker-maven-plugin`, right? If so, what should i do if i plan to add
>> `rc0-prepare.sh` for `shardingsphere/sharding-proxy` image (Not for
>> `skyapm/e2e-container` image).
>> 2. Who will call `rc0-prepare.sh` and when?
>>
>>
>> I spent much time on learning more `e2e-test` and docker image from other
>> e2e modules, which seems a blocker to me to work on sub-task 5th. Could
>> anyone give me some ideas? Thanks in advance. :)
>>
>>
>> [1] https://github.com/apache/skywalking/pull/4421
>>
>>
>> Juan Pan (Trista)
>>
>> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
>> E-mail: panjuan@apache.org
>>
>>
>>
>>
>> On 02/16/2020 18:36，Juan Pan<pa...@apache.org> wrote：
>> Hi fellows,
>>
>>
>> I am working on providing ShardingSphere[1] as one of the server storages
>> for Skywalking.
>> ShardingSphere will help shard the data collected by Skywalking into
>> different databases(Popular RDBMS, like MySQL, PostgreSQL and so on) and
>> tables to seek better query performance and do scale-out of database.
>>
>>
>> Since it is not an easy job, and needs some helps from community, i put
>> task schedule and discussion items here to welcome your comments and
>> suggestions.
>> If you are interested in this one, welcome to be my partner! :-)
>>
>>
>> ------------
>> Here is the schedule,
>> 1. Acquire all required tables and query SQLs from db logs, like general
>> log of MySQL. [DONE]
>> 2. Analyse those tables and query SQLs to explore appropriate sharding
>> strategy.[DOING]
>> 3. Provide class implementation of `PreciseShardingAlgorithm` and
>> `RangeShardingAlgorithm` of ShardingSphere to do sharding.[TODO]
>> 4. Provide configuration of Sharding Proxy[2] for Skywalking.[TODO]
>> 5. Provide Acceptance test(maybe e2e tests of Skywalking?).[TODO]
>> 6. Update readme[3].[TODO]
>> 7. Update storage doc[4].[TODO]
>> -------------
>>
>>
>> It seems a long-term work, and i will do it step by step. Any progress
>> will be updated here, also ISSUE [5] will give you thread of PRs and
>> discussion detail.
>>
>>
>> [1] https://shardingsphere.apache.org/
>> [2]
>> https://shardingsphere.apache.org/document/current/en/quick-start/sharding-proxy-quick-start/
>> [3] https://github.com/apache/skywalking/tree/master/install
>> [4 ]
>> https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-storage.md
>> [5] https://github.com/apache/skywalking/issues/4131
>>
>>
>>
>>
>>
>> Juan Pan (Trista)
>>
>> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
>> E-mail: panjuan@apache.org
>>
>>
>>
>>
Re: [DISCUSS] Add Apache ShardingSphere as the server storage for
Apache SkyWalking
Posted by Sheng Wu <wu...@gmail.com>.
Thinks happen quickly, I think that PR is going to be merged soon.
You could wait 2-3 days.
Sheng Wu 吴晟
Twitter, wusheng1108
Juan Pan <pa...@apache.org> 于2020年3月15日周日 下午3:39写道：
> Hi,
>
>
> Thanks for your kind remainder.
> Actually i am on the way writing e2e tests for ShardingSphere. Considering
> this PR[1] has not been merged and the discussion is underway,
> do you think i should stop my current work (Based on the old one) and wait
> for your progress?
>
>
> Cheers,
> Trista
>
>
> Juan Pan (Trista)
>
> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
> E-mail: panjuan@apache.org
>
>
>
>
> On 03/15/2020 14:08，kezhenxu94<ke...@apache.org> wrote：
> Hi Juan Pan, we've noticed that the E2E tests are now very unfriendly for
> new contributors, thus we have rewrote the tests now and, from my point of
> view, if you're familiar with Docker and docker compose, you should be
> quick to get started with the new E2E framework, please take a look at this
> pull request https://github.com/apache/skywalking/pull/4510 and check out
> the branch locally to play around it, and I hope to get feedback from E2E
> contributors like you whether it improves, compared with the old one,
> thanks.
>
>
>
>
>
>
> --
>
> GitHub @kezhenxu94
> Apache SkyWalking, Apache Dubbo
>
>
>
> At 2020-03-15 13:42:55, "Juan Pan" <pa...@apache.org> wrote:
> Hi community,
>
>
> Update task status and raise my questions. Could I get your bits of help?
>
>
> According to our schedule, the 1st to 3rd sub-tasks have been completed
> already[1]. Currently, I am working on sub-task 4th and 5th, i.e.,
> 4. Provide configuration of Sharding Proxy[2] for Skywalking.
> 5. Provide e2e tests.
>
>
> Here are my questions,
> 1. `rc0-prepare.sh` is used only for one `image` of `docker-maven-plugin`,
> right? If so, what should i do if i plan to add `rc0-prepare.sh` for
> `shardingsphere/sharding-proxy` image (Not for `skyapm/e2e-container`
> image).
> 2. Who will call `rc0-prepare.sh` and when?
>
>
> I spent much time on learning more `e2e-test` and docker image from other
> e2e modules, which seems a blocker to me to work on sub-task 5th. Could
> anyone give me some ideas? Thanks in advance. :)
>
>
> [1] https://github.com/apache/skywalking/pull/4421
>
>
> Juan Pan (Trista)
>
> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
> E-mail: panjuan@apache.org
>
>
>
>
> On 02/16/2020 18:36，Juan Pan<pa...@apache.org> wrote：
> Hi fellows,
>
>
> I am working on providing ShardingSphere[1] as one of the server storages
> for Skywalking.
> ShardingSphere will help shard the data collected by Skywalking into
> different databases(Popular RDBMS, like MySQL, PostgreSQL and so on) and
> tables to seek better query performance and do scale-out of database.
>
>
> Since it is not an easy job, and needs some helps from community, i put
> task schedule and discussion items here to welcome your comments and
> suggestions.
> If you are interested in this one, welcome to be my partner! :-)
>
>
> ------------
> Here is the schedule,
> 1. Acquire all required tables and query SQLs from db logs, like general
> log of MySQL. [DONE]
> 2. Analyse those tables and query SQLs to explore appropriate sharding
> strategy.[DOING]
> 3. Provide class implementation of `PreciseShardingAlgorithm` and
> `RangeShardingAlgorithm` of ShardingSphere to do sharding.[TODO]
> 4. Provide configuration of Sharding Proxy[2] for Skywalking.[TODO]
> 5. Provide Acceptance test(maybe e2e tests of Skywalking?).[TODO]
> 6. Update readme[3].[TODO]
> 7. Update storage doc[4].[TODO]
> -------------
>
>
> It seems a long-term work, and i will do it step by step. Any progress
> will be updated here, also ISSUE [5] will give you thread of PRs and
> discussion detail.
>
>
> [1] https://shardingsphere.apache.org/
> [2]
> https://shardingsphere.apache.org/document/current/en/quick-start/sharding-proxy-quick-start/
> [3] https://github.com/apache/skywalking/tree/master/install
> [4 ]
> https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-storage.md
> [5] https://github.com/apache/skywalking/issues/4131
>
>
>
>
>
> Juan Pan (Trista)
>
> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
> E-mail: panjuan@apache.org
>
>
>
>
Re:[DISCUSS] Add Apache ShardingSphere as the server storage for
Apache SkyWalking
Posted by Juan Pan <pa...@apache.org>.
Hi,
Thanks for your kind remainder.
Actually i am on the way writing e2e tests for ShardingSphere. Considering this PR[1] has not been merged and the discussion is underway,
do you think i should stop my current work (Based on the old one) and wait for your progress?
Cheers,
Trista
Juan Pan (Trista)
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: panjuan@apache.org
On 03/15/2020 14:08，kezhenxu94<ke...@apache.org> wrote：
Hi Juan Pan, we've noticed that the E2E tests are now very unfriendly for new contributors, thus we have rewrote the tests now and, from my point of view, if you're familiar with Docker and docker compose, you should be quick to get started with the new E2E framework, please take a look at this pull request https://github.com/apache/skywalking/pull/4510 and check out the branch locally to play around it, and I hope to get feedback from E2E contributors like you whether it improves, compared with the old one, thanks.
--
GitHub @kezhenxu94
Apache SkyWalking, Apache Dubbo
At 2020-03-15 13:42:55, "Juan Pan" <pa...@apache.org> wrote:
Hi community,
Update task status and raise my questions. Could I get your bits of help?
According to our schedule, the 1st to 3rd sub-tasks have been completed already[1]. Currently, I am working on sub-task 4th and 5th, i.e.,
4. Provide configuration of Sharding Proxy[2] for Skywalking.
5. Provide e2e tests.
Here are my questions,
1. `rc0-prepare.sh` is used only for one `image` of `docker-maven-plugin`, right? If so, what should i do if i plan to add `rc0-prepare.sh` for `shardingsphere/sharding-proxy` image (Not for `skyapm/e2e-container` image).
2. Who will call `rc0-prepare.sh` and when?
I spent much time on learning more `e2e-test` and docker image from other e2e modules, which seems a blocker to me to work on sub-task 5th. Could anyone give me some ideas? Thanks in advance. :)
[1] https://github.com/apache/skywalking/pull/4421
Juan Pan (Trista)
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: panjuan@apache.org
On 02/16/2020 18:36，Juan Pan<pa...@apache.org> wrote：
Hi fellows,
I am working on providing ShardingSphere[1] as one of the server storages for Skywalking.
ShardingSphere will help shard the data collected by Skywalking into different databases(Popular RDBMS, like MySQL, PostgreSQL and so on) and tables to seek better query performance and do scale-out of database.
Since it is not an easy job, and needs some helps from community, i put task schedule and discussion items here to welcome your comments and suggestions.
If you are interested in this one, welcome to be my partner! :-)
------------
Here is the schedule,
1. Acquire all required tables and query SQLs from db logs, like general log of MySQL. [DONE]
2. Analyse those tables and query SQLs to explore appropriate sharding strategy.[DOING]
3. Provide class implementation of `PreciseShardingAlgorithm` and `RangeShardingAlgorithm` of ShardingSphere to do sharding.[TODO]
4. Provide configuration of Sharding Proxy[2] for Skywalking.[TODO]
5. Provide Acceptance test(maybe e2e tests of Skywalking?).[TODO]
6. Update readme[3].[TODO]
7. Update storage doc[4].[TODO]
-------------
It seems a long-term work, and i will do it step by step. Any progress will be updated here, also ISSUE [5] will give you thread of PRs and discussion detail.
[1] https://shardingsphere.apache.org/
[2] https://shardingsphere.apache.org/document/current/en/quick-start/sharding-proxy-quick-start/
[3] https://github.com/apache/skywalking/tree/master/install
[4 ]https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-storage.md
[5] https://github.com/apache/skywalking/issues/4131
Juan Pan (Trista)
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: panjuan@apache.org
Re:Re:[DISCUSS] Add Apache ShardingSphere as the server storage for
Apache SkyWalking
Posted by kezhenxu94 <ke...@apache.org>.
Hi Juan Pan, we've noticed that the E2E tests are now very unfriendly for new contributors, thus we have rewrote the tests now and, from my point of view, if you're familiar with Docker and docker compose, you should be quick to get started with the new E2E framework, please take a look at this pull request https://github.com/apache/skywalking/pull/4510 and check out the branch locally to play around it, and I hope to get feedback from E2E contributors like you whether it improves, compared with the old one, thanks.
--
GitHub @kezhenxu94
Apache SkyWalking, Apache Dubbo
At 2020-03-15 13:42:55, "Juan Pan" <pa...@apache.org> wrote:
>Hi community,
>
>
>Update task status and raise my questions. Could I get your bits of help?
>
>
>According to our schedule, the 1st to 3rd sub-tasks have been completed already[1]. Currently, I am working on sub-task 4th and 5th, i.e.,
>4. Provide configuration of Sharding Proxy[2] for Skywalking.
>5. Provide e2e tests.
>
>
>Here are my questions,
>1. `rc0-prepare.sh` is used only for one `image` of `docker-maven-plugin`, right? If so, what should i do if i plan to add `rc0-prepare.sh` for `shardingsphere/sharding-proxy` image (Not for `skyapm/e2e-container` image).
>2. Who will call `rc0-prepare.sh` and when?
>
>
>I spent much time on learning more `e2e-test` and docker image from other e2e modules, which seems a blocker to me to work on sub-task 5th. Could anyone give me some ideas? Thanks in advance. :)
>
>
>[1] https://github.com/apache/skywalking/pull/4421
>
>
> Juan Pan (Trista)
>
>Senior DBA & PPMC of Apache ShardingSphere(Incubating)
>E-mail: panjuan@apache.org
>
>
>
>
>On 02/16/2020 18:36，Juan Pan<pa...@apache.org> wrote：
>Hi fellows,
>
>
>I am working on providing ShardingSphere[1] as one of the server storages for Skywalking.
>ShardingSphere will help shard the data collected by Skywalking into different databases(Popular RDBMS, like MySQL, PostgreSQL and so on) and tables to seek better query performance and do scale-out of database.
>
>
>Since it is not an easy job, and needs some helps from community, i put task schedule and discussion items here to welcome your comments and suggestions.
>If you are interested in this one, welcome to be my partner! :-)
>
>
>------------
>Here is the schedule,
>1. Acquire all required tables and query SQLs from db logs, like general log of MySQL. [DONE]
>2. Analyse those tables and query SQLs to explore appropriate sharding strategy.[DOING]
>3. Provide class implementation of `PreciseShardingAlgorithm` and `RangeShardingAlgorithm` of ShardingSphere to do sharding.[TODO]
>4. Provide configuration of Sharding Proxy[2] for Skywalking.[TODO]
>5. Provide Acceptance test(maybe e2e tests of Skywalking?).[TODO]
>6. Update readme[3].[TODO]
>7. Update storage doc[4].[TODO]
>-------------
>
>
>It seems a long-term work, and i will do it step by step. Any progress will be updated here, also ISSUE [5] will give you thread of PRs and discussion detail.
>
>
>[1] https://shardingsphere.apache.org/
>[2] https://shardingsphere.apache.org/document/current/en/quick-start/sharding-proxy-quick-start/
>[3] https://github.com/apache/skywalking/tree/master/install
>[4 ]https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-storage.md
>[5] https://github.com/apache/skywalking/issues/4131
>
>
>
>
>
> Juan Pan (Trista)
>
>Senior DBA & PPMC of Apache ShardingSphere(Incubating)
>E-mail: panjuan@apache.org
>
>
>
Re:[DISCUSS] Add Apache ShardingSphere as the server storage for
Apache SkyWalking
Posted by Juan Pan <pa...@apache.org>.
Hi community,
Update task status and raise my questions. Could I get your bits of help?
According to our schedule, the 1st to 3rd sub-tasks have been completed already[1]. Currently, I am working on sub-task 4th and 5th, i.e.,
4. Provide configuration of Sharding Proxy[2] for Skywalking.
5. Provide e2e tests.
Here are my questions,
1. `rc0-prepare.sh` is used only for one `image` of `docker-maven-plugin`, right? If so, what should i do if i plan to add `rc0-prepare.sh` for `shardingsphere/sharding-proxy` image (Not for `skyapm/e2e-container` image).
2. Who will call `rc0-prepare.sh` and when?
I spent much time on learning more `e2e-test` and docker image from other e2e modules, which seems a blocker to me to work on sub-task 5th. Could anyone give me some ideas? Thanks in advance. :)
[1] https://github.com/apache/skywalking/pull/4421
Juan Pan (Trista)
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: panjuan@apache.org
On 02/16/2020 18:36，Juan Pan<pa...@apache.org> wrote：
Hi fellows,
I am working on providing ShardingSphere[1] as one of the server storages for Skywalking.
ShardingSphere will help shard the data collected by Skywalking into different databases(Popular RDBMS, like MySQL, PostgreSQL and so on) and tables to seek better query performance and do scale-out of database.
Since it is not an easy job, and needs some helps from community, i put task schedule and discussion items here to welcome your comments and suggestions.
If you are interested in this one, welcome to be my partner! :-)
------------
Here is the schedule,
1. Acquire all required tables and query SQLs from db logs, like general log of MySQL. [DONE]
2. Analyse those tables and query SQLs to explore appropriate sharding strategy.[DOING]
3. Provide class implementation of `PreciseShardingAlgorithm` and `RangeShardingAlgorithm` of ShardingSphere to do sharding.[TODO]
4. Provide configuration of Sharding Proxy[2] for Skywalking.[TODO]
5. Provide Acceptance test(maybe e2e tests of Skywalking?).[TODO]
6. Update readme[3].[TODO]
7. Update storage doc[4].[TODO]
-------------
It seems a long-term work, and i will do it step by step. Any progress will be updated here, also ISSUE [5] will give you thread of PRs and discussion detail.
[1] https://shardingsphere.apache.org/
[2] https://shardingsphere.apache.org/document/current/en/quick-start/sharding-proxy-quick-start/
[3] https://github.com/apache/skywalking/tree/master/install
[4 ]https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-storage.md
[5] https://github.com/apache/skywalking/issues/4131
Juan Pan (Trista)
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: panjuan@apache.org
Re: [DISCUSS] Add Apache ShardingSphere as the server storage for
Apache SkyWalking
Posted by Juan Pan <pa...@apache.org>.
Make sense to me, thanks for your reply.
Juan Pan (Trista)
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: panjuan@apache.org
On 02/16/2020 22:20，Sheng Wu<wu...@gmail.com> wrote：
Juan Pan <pa...@apache.org> 于2020年2月16日周日 下午6:36写道：
Hi fellows,
I am working on providing ShardingSphere[1] as one of the server storages
for Skywalking.
ShardingSphere will help shard the data collected by Skywalking into
different databases(Popular RDBMS, like MySQL, PostgreSQL and so on) and
tables to seek better query performance and do scale-out of database.
Since it is not an easy job, and needs some helps from community, i put
task schedule and discussion items here to welcome your comments and
suggestions.
If you are interested in this one, welcome to be my partner! :-)
------------
Here is the schedule,
1. Acquire all required tables and query SQLs from db logs, like general
log of MySQL. [DONE]
2. Analyse those tables and query SQLs to explore appropriate sharding
strategy.[DOING]
3. Provide class implementation of `PreciseShardingAlgorithm` and
`RangeShardingAlgorithm` of ShardingSphere to do sharding.[TODO]
4. Provide configuration of Sharding Proxy[2] for Skywalking.[TODO]
5. Provide Acceptance test(maybe e2e tests of Skywalking?).[TODO
Yes. We call it e2e. You could find many storage implementation tests
there. Zhenxu Ke and Haochao Zhuang are the two leading this part.
BTW, your plan looks good to me.
Sheng Wu 吴晟
Twitter, wusheng1108
]
6. Update readme[3].[TODO]
7. Update storage doc[4].[TODO]
-------------
It seems a long-term work, and i will do it step by step. Any progress
will be updated here, also ISSUE [5] will give you thread of PRs and
discussion detail.
[1] https://shardingsphere.apache.org/
[2]
https://shardingsphere.apache.org/document/current/en/quick-start/sharding-proxy-quick-start/
[3] https://github.com/apache/skywalking/tree/master/install
[4 ]
https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-storage.md
[5] https://github.com/apache/skywalking/issues/4131
Juan Pan (Trista)
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: panjuan@apache.org
Re: [DISCUSS] Add Apache ShardingSphere as the server storage for
Apache SkyWalking
Posted by Sheng Wu <wu...@gmail.com>.
Juan Pan <pa...@apache.org> 于2020年2月16日周日 下午6:36写道：
> Hi fellows,
>
>
> I am working on providing ShardingSphere[1] as one of the server storages
> for Skywalking.
> ShardingSphere will help shard the data collected by Skywalking into
> different databases(Popular RDBMS, like MySQL, PostgreSQL and so on) and
> tables to seek better query performance and do scale-out of database.
>
>
> Since it is not an easy job, and needs some helps from community, i put
> task schedule and discussion items here to welcome your comments and
> suggestions.
> If you are interested in this one, welcome to be my partner! :-)
>
>
> ------------
> Here is the schedule,
> 1. Acquire all required tables and query SQLs from db logs, like general
> log of MySQL. [DONE]
> 2. Analyse those tables and query SQLs to explore appropriate sharding
> strategy.[DOING]
> 3. Provide class implementation of `PreciseShardingAlgorithm` and
> `RangeShardingAlgorithm` of ShardingSphere to do sharding.[TODO]
> 4. Provide configuration of Sharding Proxy[2] for Skywalking.[TODO]
5. Provide Acceptance test(maybe e2e tests of Skywalking?).[TODO
Yes. We call it e2e. You could find many storage implementation tests
there. Zhenxu Ke and Haochao Zhuang are the two leading this part.
BTW, your plan looks good to me.
Sheng Wu 吴晟
Twitter, wusheng1108
> ]
> 6. Update readme[3].[TODO]
> 7. Update storage doc[4].[TODO]
> -------------
>
>
> It seems a long-term work, and i will do it step by step. Any progress
> will be updated here, also ISSUE [5] will give you thread of PRs and
> discussion detail.
>
>
> [1] https://shardingsphere.apache.org/
> [2]
> https://shardingsphere.apache.org/document/current/en/quick-start/sharding-proxy-quick-start/
> [3] https://github.com/apache/skywalking/tree/master/install
> [4 ]
> https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-storage.md
> [5] https://github.com/apache/skywalking/issues/4131
>
>
>
>
>
> Juan Pan (Trista)
>
> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
> E-mail: panjuan@apache.org
>
>
>
>