Re: [DISCUSS] Add Apache ShardingSphere as the server storage for Apache SkyWalking

[Sheng Wu <wu...@gmail.com>]

Hi Juan Pan

I am updating the 8.0.0 core in the PR[1].
I removed the `-day`, `-month`, `-hour` tables. And *_inventory tables have
been removed, with several metadata tables.

Please recheck the storage, the config file is required to change.

[1] https://github.com/apache/skywalking/pull/4599

Sheng Wu 吴晟
Twitter, wusheng1108


Sheng Wu <wu...@gmail.com> 于2020年3月16日周一 上午8:57写道：

> Thinks happen quickly, I think that PR is going to be merged soon.
> You could wait 2-3 days.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
>
> Juan Pan <pa...@apache.org> 于2020年3月15日周日 下午3:39写道：
>
>> Hi,
>>
>>
>> Thanks for your kind remainder.
>> Actually i am on the way writing e2e tests for ShardingSphere.
>> Considering this PR[1] has not been merged and the discussion is underway,
>> do you think i should stop my current work (Based on the old one) and
>> wait for your progress?
>>
>>
>> Cheers,
>> Trista
>>
>>
>>  Juan Pan (Trista)
>>
>> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
>> E-mail: panjuan@apache.org
>>
>>
>>
>>
>> On 03/15/2020 14:08，kezhenxu94<ke...@apache.org> wrote：
>> Hi Juan Pan, we've noticed that the E2E tests are now very unfriendly for
>> new contributors, thus we have rewrote the tests now and, from my point of
>> view, if you're familiar with Docker and docker compose, you should be
>> quick to get started with the new E2E framework, please take a look at this
>> pull request https://github.com/apache/skywalking/pull/4510 and check
>> out the branch locally to play around it, and I hope to get feedback from
>> E2E contributors like you whether it improves, compared with the old one,
>> thanks.
>>
>>
>>
>>
>>
>>
>> --
>>
>> GitHub @kezhenxu94
>> Apache SkyWalking, Apache Dubbo
>>
>>
>>
>> At 2020-03-15 13:42:55, "Juan Pan" <pa...@apache.org> wrote:
>> Hi community,
>>
>>
>> Update task status and raise my questions. Could I get your bits of help?
>>
>>
>> According to our schedule, the 1st to 3rd sub-tasks have been completed
>> already[1]. Currently, I am working on sub-task 4th and 5th, i.e.,
>> 4. Provide configuration of Sharding Proxy[2] for Skywalking.
>> 5. Provide e2e tests.
>>
>>
>> Here are my questions,
>> 1. `rc0-prepare.sh` is used only for one `image` of
>> `docker-maven-plugin`, right? If so, what should i do if i plan to add
>> `rc0-prepare.sh` for `shardingsphere/sharding-proxy` image (Not for
>> `skyapm/e2e-container` image).
>> 2. Who will call `rc0-prepare.sh` and when?
>>
>>
>> I spent much time on learning more `e2e-test` and docker image from other
>> e2e modules, which seems a blocker to me to work on sub-task 5th. Could
>> anyone give me some ideas? Thanks in advance. :)
>>
>>
>> [1] https://github.com/apache/skywalking/pull/4421
>>
>>
>> Juan Pan (Trista)
>>
>> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
>> E-mail: panjuan@apache.org
>>
>>
>>
>>
>> On 02/16/2020 18:36，Juan Pan<pa...@apache.org> wrote：
>> Hi fellows,
>>
>>
>> I am working on providing ShardingSphere[1] as one of the server storages
>> for Skywalking.
>> ShardingSphere will help shard the data collected by Skywalking into
>> different databases(Popular RDBMS, like MySQL, PostgreSQL and so on) and
>> tables to seek better query performance and do scale-out of database.
>>
>>
>> Since it is not an easy job, and needs some helps from community, i put
>> task schedule and discussion items here to welcome your comments and
>> suggestions.
>> If you are interested in this one, welcome to be my partner! :-)
>>
>>
>> ------------
>> Here is the schedule,
>> 1. Acquire all required tables and query SQLs from db logs, like general
>> log of MySQL. [DONE]
>> 2. Analyse those tables and query SQLs to explore appropriate sharding
>> strategy.[DOING]
>> 3. Provide class implementation of `PreciseShardingAlgorithm` and
>> `RangeShardingAlgorithm` of ShardingSphere to do sharding.[TODO]
>> 4. Provide configuration of Sharding Proxy[2] for Skywalking.[TODO]
>> 5. Provide Acceptance test(maybe e2e tests of Skywalking?).[TODO]
>> 6. Update readme[3].[TODO]
>> 7. Update storage doc[4].[TODO]
>> -------------
>>
>>
>> It seems a long-term work, and i will do it step by step. Any progress
>> will be updated here, also ISSUE [5] will give you thread of PRs and
>> discussion detail.
>>
>>
>> [1] https://shardingsphere.apache.org/
>> [2]
>> https://shardingsphere.apache.org/document/current/en/quick-start/sharding-proxy-quick-start/
>> [3] https://github.com/apache/skywalking/tree/master/install
>> [4 ]
>> https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-storage.md
>> [5] https://github.com/apache/skywalking/issues/4131
>>
>>
>>
>>
>>
>> Juan Pan (Trista)
>>
>> Senior DBA & PPMC of Apache ShardingSphere(Incubating)
>> E-mail: panjuan@apache.org
>>
>>
>>
>>