You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/04/13 11:15:11 UTC
[GitHub] [incubator-shardingsphere] menghaoranss opened a new pull request
#5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
menghaoranss opened a new pull request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
URL: https://github.com/apache/incubator-shardingsphere/pull/5166
For #5128 .
Changes proposed in this pull request:
- Add metadata center documents
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] kimmking commented on a change in pull
request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
Posted by GitBox <gi...@apache.org>.
kimmking commented on a change in pull request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
URL: https://github.com/apache/incubator-shardingsphere/pull/5166#discussion_r407438983
##########
File path: docs/document/content/features/orchestration/_index.en.md
##########
@@ -7,4 +7,4 @@ chapter = true
## Navigation
-Provide orchestration of config/registry center(and metadata center in plan), dynamic configuration, database & application disabling, and APM.
+Provide orchestration of config/registry center/metadata center, dynamic configuration, database & application disabling, and APM.
Review comment:
config/registry/metadata center
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] kimmking commented on a change in pull
request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
Posted by GitBox <gi...@apache.org>.
kimmking commented on a change in pull request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
URL: https://github.com/apache/incubator-shardingsphere/pull/5166#discussion_r407440616
##########
File path: docs/document/content/features/orchestration/metadata-center.cn.md
##########
@@ -0,0 +1,103 @@
++++
+pre = "<b>3.3.3. </b>"
+toc = true
+title = "元数据中心"
+weight = 3
++++
+
+## 实现动机
+
+- 元数据是Sharding-JDBC/Sharding-Proxy所使用的数据源的表、列和索引等核心数据，元数据保障ShardingSphere各个组件能够正确运行。
+
+- 元数据中心对元数据进行统一组织和管理，实现元数据的统一加载、变更通知和数据同步。
+
+## 元数据中心数据结构
+
+元数据中心在定义的命名空间的metadata节点下，以YAML格式存储，每个逻辑数据源独立存储。
+
+```
Review comment:
missing namespace&orchestration node？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] menghaoranss commented on a change in
pull request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
Posted by GitBox <gi...@apache.org>.
menghaoranss commented on a change in pull request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
URL: https://github.com/apache/incubator-shardingsphere/pull/5166#discussion_r407457052
##########
File path: docs/document/content/features/orchestration/metadata-center.cn.md
##########
@@ -0,0 +1,103 @@
++++
+pre = "<b>3.3.3. </b>"
+toc = true
+title = "元数据中心"
+weight = 3
++++
+
+## 实现动机
+
+- 元数据是Sharding-JDBC/Sharding-Proxy所使用的数据源的表、列和索引等核心数据，元数据保障ShardingSphere各个组件能够正确运行。
+
+- 元数据中心对元数据进行统一组织和管理，实现元数据的统一加载、变更通知和数据同步。
+
+## 元数据中心数据结构
+
+元数据中心在定义的命名空间的metadata节点下，以YAML格式存储，每个逻辑数据源独立存储。
+
+```
+metadata
+ ├──schema1
+ ├ ├── [YAML text contents]
+ ├──schema2
+ ├ ├── [YAML text contents]
+ ├──....
+```
+
+## YAML Text Contents
+
+在元数据中心中，元数据分为 `configuredSchemaMetaData` 和 `unconfiguredSchemaMetaDataMap` 两部分。
+
+元数据内容目前不支持动态修改。
+
+```
+configuredSchemaMetaData:
+ tables: # 表
+ t_order: # 表名
+ columns: # 列
+ id: # 列名
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: id
+ primaryKey: trues
+ order_id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: order_id
+ primaryKey: false
+ indexs: # 索引
+ t_user_order_id_index: # 索引名
+ name: t_user_order_id_index
+ t_order_item:
+ columns:
+ order_id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: order_id
+ primaryKey: false
+unconfiguredSchemaMetaDataMap:
+ ds_0: # 数据源
+ tables: # 表
+ t_user: # 表名
+ columns: # 列
+ user_id: # 列名
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: user_id
+ primaryKey: false
+ id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: id
+ primaryKey: true
+ order_id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: order_id
+ primaryKey: false
+ indexes: # 索引
+ t_user_order_id_index: # 索引名
+ name: t_user_order_id_index
+ t_user_user_id_index:
+ name: t_user_user_id_index
+ primary:
+ name: PRIMARY
+```
+
+### configuredSchemaMetaData
+
+存储所有配置了分片规则的数据源的元数据。
+
+### unconfiguredSchemaMetaDataMap
+
+存储没有配置分片规则的数据源的元数据。
+
+## 变更通知
+
+通过某一个Proxy实例执行DDL以后，元数据中心通过事件机制，通知其它Proxy实例重新加载元数据，保证元数据一致。
Review comment:
系统 -> ShardingSphere maybe better.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] coveralls commented on issue #5166:
Optimize MetaData Center #MetaDataCenter-phase-3-4
Posted by GitBox <gi...@apache.org>.
coveralls commented on issue #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
URL: https://github.com/apache/incubator-shardingsphere/pull/5166#issuecomment-612875336
## Pull Request Test Coverage Report for [Build 11005](https://coveralls.io/builds/30037723)
* **0** of **0** changed or added relevant lines in **0** files are covered.
* No unchanged relevant lines lost coverage.
* Overall coverage remained the same at **57.167%**
---
| Totals | [](https://coveralls.io/builds/30037723) |
| :-- | --: |
| Change from base [Build 10823](https://coveralls.io/builds/29820781): | 0.0% |
| Covered Lines: | 11709 |
| Relevant Lines: | 20482 |
---
##### 💛 - [Coveralls](https://coveralls.io)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] kimmking commented on a change in pull
request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
Posted by GitBox <gi...@apache.org>.
kimmking commented on a change in pull request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
URL: https://github.com/apache/incubator-shardingsphere/pull/5166#discussion_r407440689
##########
File path: docs/document/content/features/orchestration/metadata-center.en.md
##########
@@ -0,0 +1,103 @@
++++
+pre = "<b>3.3.3. </b>"
+toc = true
+title = "Metadata Center"
+weight = 3
++++
+
+## Motivation
+
+- Metadata is the core data of the data source used by Sharding-JDBC / Sharding-Proxy, which contains tables, columns, and indexes, etc. Metadata ensures that each component of ShardingSphere can run correctly.
+
+- The metadata center organizes and manages the metadata in a unified manner to realize the unified loading of metadata, change notifications and data synchronization.
+
+## Data Structure in Metadata Center
+
+The metadata center stores metadata in YAML under the metadata node of the defined namespace, and each logical data source is stored independently.
+
+```
+metadata
+ ├──schema1
+ ├ ├── [YAML text contents]
+ ├──schema2
+ ├ ├── [YAML text contents]
+ ├──....
+```
+
+## YAML text contents
+
+In the metadata center, metadata is divided into two parts: `configuredSchemaMetaData` and` unconfiguredSchemaMetaDataMap`.
+
+Dynamic modification of metadata content is not supported currently.
+
+```
+configuredSchemaMetaData:
+ tables: # Tables
+ t_order: # table_name
+ columns: # Columns
+ id: # column_name
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: id
+ primaryKey: trues
+ order_id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: order_id
+ primaryKey: false
+ indexs: # Indexes
+ t_user_order_id_index: # index_name
+ name: t_user_order_id_index
+ t_order_item:
+ columns:
+ order_id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: order_id
+ primaryKey: false
+unconfiguredSchemaMetaDataMap:
+ ds_0: # DataSources
+ tables: # Tables
+ t_user: # table_name
+ columns: # Columns
+ user_id: # column_name
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: user_id
+ primaryKey: false
+ id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: id
+ primaryKey: true
+ order_id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: order_id
+ primaryKey: false
+ indexes: # Indexes
+ t_user_order_id_index: # index_name
+ name: t_user_order_id_index
+ t_user_user_id_index:
+ name: t_user_user_id_index
+ primary:
+ name: PRIMARY
+```
+
+### configuredSchemaMetaData
+
+Store metadata for all data sources configured with sharding rules.
+
+### unconfiguredSchemaMetaDataMap
+
+Store metadata for data sources that no sharding rules configured.
+
+## Change Notifications
+
+After DDL is executed through a certain Proxy instance, the metadata center notifies other Proxy instances to reload metadata through an event mechanism to ensure metadata consistency.
Review comment:
as the above
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] kimmking commented on a change in pull
request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
Posted by GitBox <gi...@apache.org>.
kimmking commented on a change in pull request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
URL: https://github.com/apache/incubator-shardingsphere/pull/5166#discussion_r407440423
##########
File path: docs/document/content/features/orchestration/metadata-center.en.md
##########
@@ -0,0 +1,103 @@
++++
+pre = "<b>3.3.3. </b>"
+toc = true
+title = "Metadata Center"
+weight = 3
++++
+
+## Motivation
+
+- Metadata is the core data of the data source used by Sharding-JDBC / Sharding-Proxy, which contains tables, columns, and indexes, etc. Metadata ensures that each component of ShardingSphere can run correctly.
Review comment:
Sharding-JDBC/Sharding-Proxy
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] kimmking merged pull request #5166:
Optimize MetaData Center #MetaDataCenter-phase-3-4
Posted by GitBox <gi...@apache.org>.
kimmking merged pull request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
URL: https://github.com/apache/incubator-shardingsphere/pull/5166
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] kimmking commented on a change in pull
request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
Posted by GitBox <gi...@apache.org>.
kimmking commented on a change in pull request #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
URL: https://github.com/apache/incubator-shardingsphere/pull/5166#discussion_r407440285
##########
File path: docs/document/content/features/orchestration/metadata-center.cn.md
##########
@@ -0,0 +1,103 @@
++++
+pre = "<b>3.3.3. </b>"
+toc = true
+title = "元数据中心"
+weight = 3
++++
+
+## 实现动机
+
+- 元数据是Sharding-JDBC/Sharding-Proxy所使用的数据源的表、列和索引等核心数据，元数据保障ShardingSphere各个组件能够正确运行。
+
+- 元数据中心对元数据进行统一组织和管理，实现元数据的统一加载、变更通知和数据同步。
+
+## 元数据中心数据结构
+
+元数据中心在定义的命名空间的metadata节点下，以YAML格式存储，每个逻辑数据源独立存储。
+
+```
+metadata
+ ├──schema1
+ ├ ├── [YAML text contents]
+ ├──schema2
+ ├ ├── [YAML text contents]
+ ├──....
+```
+
+## YAML Text Contents
+
+在元数据中心中，元数据分为 `configuredSchemaMetaData` 和 `unconfiguredSchemaMetaDataMap` 两部分。
+
+元数据内容目前不支持动态修改。
+
+```
+configuredSchemaMetaData:
+ tables: # 表
+ t_order: # 表名
+ columns: # 列
+ id: # 列名
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: id
+ primaryKey: trues
+ order_id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: order_id
+ primaryKey: false
+ indexs: # 索引
+ t_user_order_id_index: # 索引名
+ name: t_user_order_id_index
+ t_order_item:
+ columns:
+ order_id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: order_id
+ primaryKey: false
+unconfiguredSchemaMetaDataMap:
+ ds_0: # 数据源
+ tables: # 表
+ t_user: # 表名
+ columns: # 列
+ user_id: # 列名
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: user_id
+ primaryKey: false
+ id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: id
+ primaryKey: true
+ order_id:
+ caseSensitive: false
+ dataType: 0
+ generated: false
+ name: order_id
+ primaryKey: false
+ indexes: # 索引
+ t_user_order_id_index: # 索引名
+ name: t_user_order_id_index
+ t_user_user_id_index:
+ name: t_user_user_id_index
+ primary:
+ name: PRIMARY
+```
+
+### configuredSchemaMetaData
+
+存储所有配置了分片规则的数据源的元数据。
+
+### unconfiguredSchemaMetaDataMap
+
+存储没有配置分片规则的数据源的元数据。
+
+## 变更通知
+
+通过某一个Proxy实例执行DDL以后，元数据中心通过事件机制，通知其它Proxy实例重新加载元数据，保证元数据一致。
Review comment:
通过某一个Proxy实例执行DDL以后，系统先将新的元数据存储到元数据中心，然后通过事件广播机制，通知其它Proxy实例从源数据库中心同步元数据，保证元数据一致。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] coveralls edited a comment on issue
#5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on issue #5166: Optimize MetaData Center #MetaDataCenter-phase-3-4
URL: https://github.com/apache/incubator-shardingsphere/pull/5166#issuecomment-612875336
## Pull Request Test Coverage Report for [Build 11012](https://coveralls.io/builds/30040440)
* **0** of **0** changed or added relevant lines in **0** files are covered.
* No unchanged relevant lines lost coverage.
* Overall coverage remained the same at **57.167%**
---
| Totals | [](https://coveralls.io/builds/30040440) |
| :-- | --: |
| Change from base [Build 10823](https://coveralls.io/builds/29820781): | 0.0% |
| Covered Lines: | 11709 |
| Relevant Lines: | 20482 |
---
##### 💛 - [Coveralls](https://coveralls.io)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services