You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by GitBox <gi...@apache.org> on 2022/10/28 12:15:39 UTC
[GitHub] [incubator-devlake-website] mappjzc opened a new pull request, #308: Project framework
mappjzc opened a new pull request, #308:
URL: https://github.com/apache/incubator-devlake-website/pull/308
# Summary
Project - framework
### Does this close any open issues?
Closes xx
### Screenshots
Include any relevant screenshots here.
### Other Information
Any other information that is important to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] Startrekzky commented on a diff in pull request #308: WIP Project framework
Posted by GitBox <gi...@apache.org>.
Startrekzky commented on code in PR #308:
URL: https://github.com/apache/incubator-devlake-website/pull/308#discussion_r1010279548
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+---
+
+## Summary
+`Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## project
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `describe` | longtext | | describe for project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of test project | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
Review Comment:
text change:
'this is one of test project' to 'this is one of the test projects'
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+---
+
+## Summary
+`Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## project
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `describe` | longtext | | describe for project | |
Review Comment:
Shall we use `description` and `description of the project`?
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+---
+
+## Summary
+`Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## project
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `describe` | longtext | | describe for project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of test project | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** |
+| ---------------- | --------------- | ---------------------- |
+| project_1 | gitlab | {'enable':true} |
+| project_2 | gitlab | {'enable':disable} |
Review Comment:
Can project_2 map to another plugin, like github?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] klesh commented on a diff in pull request #308: WIP Project framework
Posted by GitBox <gi...@apache.org>.
klesh commented on code in PR #308:
URL: https://github.com/apache/incubator-devlake-website/pull/308#discussion_r1011469850
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
Review Comment:
`project_metrics` describes what metrics a project had enabled.
"both side of the mapping" doesn't sound right, it feels like you are talking about `project_mapping`, please remove this part to avoid confusion.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
+
+当我们创建出 `project` 对象之后：
+
+- 我们可以通过 `GET` `/project` 接口来获取具体的的某一个 `project`的信息，除了名称外，这些信息里也包含了project的更新时间和创建时间。
Review Comment:
should return the `project_metrics` and `project_mapping` information for the project as well
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
+
+当我们创建出 `project` 对象之后：
+
+- 我们可以通过 `GET` `/project` 接口来获取具体的的某一个 `project`的信息，除了名称外，这些信息里也包含了project的更新时间和创建时间。
+- 我们可以通过 `UPDATE` `/project` 接口来更新具体的某一个 `project`的信息，比如更新其相关的 `描述` 信息。
+- 我们可以通过 `GET` `/projects` 接口来获取所有的`project`信息。
+
+2. 与此同时，我们可以通过 `GET` `/plugininfo` 接口来获取到 完整详细的 `插件` 信息，其中包括对应 `插件` 的每个表的结构信息。除此以外，我们也可以通过调用相对简单的 `GET` `/plugins` 来获取更为简洁的插件信息。简洁的插件信息仅包含插件名称，也就是`plugin_name`组成的列表，除此之外不包含其他任何信息。
+
+3. 在上述过程之后 通过 `POST` `/project_metrics` 接口来创建一组 `project` 与 `plugins` 之间的关系。使用我们前面获取到的 `project_name` 和 `plugin_name` 来构建这种关系。同时为其配置相应的 `option`.
Review Comment:
I think a better way is to accept `project_metrics` and `project_mapping` in `POST /projects` and `PATCH /projects/:projectId` endpoint, so that users may create them in one request. which makes more sense to me. @mintsweet what do you think?
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
+
+当我们创建出 `project` 对象之后：
+
+- 我们可以通过 `GET` `/project` 接口来获取具体的的某一个 `project`的信息，除了名称外，这些信息里也包含了project的更新时间和创建时间。
+- 我们可以通过 `UPDATE` `/project` 接口来更新具体的某一个 `project`的信息，比如更新其相关的 `描述` 信息。
Review Comment:
`PATCH /projects/:projectId`
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
+
+当我们创建出 `project` 对象之后：
+
+- 我们可以通过 `GET` `/project` 接口来获取具体的的某一个 `project`的信息，除了名称外，这些信息里也包含了project的更新时间和创建时间。
+- 我们可以通过 `UPDATE` `/project` 接口来更新具体的某一个 `project`的信息，比如更新其相关的 `描述` 信息。
+- 我们可以通过 `GET` `/projects` 接口来获取所有的`project`信息。
+
+2. 与此同时，我们可以通过 `GET` `/plugininfo` 接口来获取到 完整详细的 `插件` 信息，其中包括对应 `插件` 的每个表的结构信息。除此以外，我们也可以通过调用相对简单的 `GET` `/plugins` 来获取更为简洁的插件信息。简洁的插件信息仅包含插件名称，也就是`plugin_name`组成的列表，除此之外不包含其他任何信息。
+
+3. 在上述过程之后 通过 `POST` `/project_metrics` 接口来创建一组 `project` 与 `plugins` 之间的关系。使用我们前面获取到的 `project_name` 和 `plugin_name` 来构建这种关系。同时为其配置相应的 `option`.
+
+当我们创建出一组 `project_metrics` 的关系后我们就可以做更进一步的操作：
+
+- 我们可以通过 `GET` `/project_metrics` 接口来获取一组 `project_metrics` 信息，我们可以只设置 `project_name` 或只设置 `plugin_name` 来进行筛选。也可以同时设置 `project_name` 和 `plugin_name` 此时我们将获取到一个特定的 `project_metrics` 信息。
+- 我们可以通过 `UPDATE` `/project_metrics` 接口来更新特定的一组 `project_metrics` 信息。这要求我们必须同时设置好 `project_name` 和 `plugin_name`。
+
+# 关于 Project 和 Blueprint
+
+- 在 `blueprint` 的表中现在添加了一个与 `project` 相关联的字段 `project_name`，该字段表示 当前的 `blueprint` 唯一的属于某一张特定的 `project`
+- 一个`project`允许被多个 `blueprint` 的 `project_name` 字段指定。也就是`project`对`blueprint`是一对多关系
+- 可以通过以下sql 快速的查看 某个 `project` 所对应的全部 `blueprint`
+
+```
+SELECT * from `_devlake_blueprints` where `project_name`="ProjectName";
+```
+
+
+## The PluginMetric Interface
+
+
+```go
+type PluginMetric interface {
+ // returns a list of required data entities and expected features.
+ // [{ "model": "cicd_tasks", "requiredFields": {"column": "type", "execptedValue": "Deployment"}}, ...]
+ RequiredDataEntities() (data map[string]interface{},err errors.Error)
+
+ // This method returns all models of the current plugin
+ GetTablesInfo() []core.Tabler
+
+ // returns if the metric depends on Project for calculation.
+ // Currently, only dora would return true.
+ IsProjectMetric() bool
+
+ // indicates which plugins must be executed before executing this one.
+ // declare a set of dependencies with this
+ RunAfter() (PluginsNames []string,errors.Error)
+
+ // returns an empty pointer of the plugin setting struct.
+ // (no concrete usage at this point)
+ Settings() (p interface{})
+}
+```
+
+## models
Review Comment:
I don't think we need this part.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
+
+当我们创建出 `project` 对象之后：
+
+- 我们可以通过 `GET` `/project` 接口来获取具体的的某一个 `project`的信息，除了名称外，这些信息里也包含了project的更新时间和创建时间。
Review Comment:
and it should be `/projects`, plural
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
Review Comment:
Please join the METHOD and URL, no need to separate them.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
Review Comment:
it doesn't seem right, we had this table already, it should contains `project_name` `table` and `row_id`
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
+
+当我们创建出 `project` 对象之后：
+
+- 我们可以通过 `GET` `/project` 接口来获取具体的的某一个 `project`的信息，除了名称外，这些信息里也包含了project的更新时间和创建时间。
+- 我们可以通过 `UPDATE` `/project` 接口来更新具体的某一个 `project`的信息，比如更新其相关的 `描述` 信息。
+- 我们可以通过 `GET` `/projects` 接口来获取所有的`project`信息。
+
+2. 与此同时，我们可以通过 `GET` `/plugininfo` 接口来获取到 完整详细的 `插件` 信息，其中包括对应 `插件` 的每个表的结构信息。除此以外，我们也可以通过调用相对简单的 `GET` `/plugins` 来获取更为简洁的插件信息。简洁的插件信息仅包含插件名称，也就是`plugin_name`组成的列表，除此之外不包含其他任何信息。
+
+3. 在上述过程之后 通过 `POST` `/project_metrics` 接口来创建一组 `project` 与 `plugins` 之间的关系。使用我们前面获取到的 `project_name` 和 `plugin_name` 来构建这种关系。同时为其配置相应的 `option`.
+
+当我们创建出一组 `project_metrics` 的关系后我们就可以做更进一步的操作：
+
+- 我们可以通过 `GET` `/project_metrics` 接口来获取一组 `project_metrics` 信息，我们可以只设置 `project_name` 或只设置 `plugin_name` 来进行筛选。也可以同时设置 `project_name` 和 `plugin_name` 此时我们将获取到一个特定的 `project_metrics` 信息。
+- 我们可以通过 `UPDATE` `/project_metrics` 接口来更新特定的一组 `project_metrics` 信息。这要求我们必须同时设置好 `project_name` 和 `plugin_name`。
+
+# 关于 Project 和 Blueprint
+
+- 在 `blueprint` 的表中现在添加了一个与 `project` 相关联的字段 `project_name`，该字段表示 当前的 `blueprint` 唯一的属于某一张特定的 `project`
+- 一个`project`允许被多个 `blueprint` 的 `project_name` 字段指定。也就是`project`对`blueprint`是一对多关系
+- 可以通过以下sql 快速的查看 某个 `project` 所对应的全部 `blueprint`
Review Comment:
no need to mention this
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
Review Comment:
and i think we should add a new item `project_mapping`
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
+
+当我们创建出 `project` 对象之后：
+
+- 我们可以通过 `GET` `/project` 接口来获取具体的的某一个 `project`的信息，除了名称外，这些信息里也包含了project的更新时间和创建时间。
+- 我们可以通过 `UPDATE` `/project` 接口来更新具体的某一个 `project`的信息，比如更新其相关的 `描述` 信息。
+- 我们可以通过 `GET` `/projects` 接口来获取所有的`project`信息。
+
+2. 与此同时，我们可以通过 `GET` `/plugininfo` 接口来获取到 完整详细的 `插件` 信息，其中包括对应 `插件` 的每个表的结构信息。除此以外，我们也可以通过调用相对简单的 `GET` `/plugins` 来获取更为简洁的插件信息。简洁的插件信息仅包含插件名称，也就是`plugin_name`组成的列表，除此之外不包含其他任何信息。
+
+3. 在上述过程之后 通过 `POST` `/project_metrics` 接口来创建一组 `project` 与 `plugins` 之间的关系。使用我们前面获取到的 `project_name` 和 `plugin_name` 来构建这种关系。同时为其配置相应的 `option`.
+
+当我们创建出一组 `project_metrics` 的关系后我们就可以做更进一步的操作：
+
+- 我们可以通过 `GET` `/project_metrics` 接口来获取一组 `project_metrics` 信息，我们可以只设置 `project_name` 或只设置 `plugin_name` 来进行筛选。也可以同时设置 `project_name` 和 `plugin_name` 此时我们将获取到一个特定的 `project_metrics` 信息。
+- 我们可以通过 `UPDATE` `/project_metrics` 接口来更新特定的一组 `project_metrics` 信息。这要求我们必须同时设置好 `project_name` 和 `plugin_name`。
+
+# 关于 Project 和 Blueprint
+
+- 在 `blueprint` 的表中现在添加了一个与 `project` 相关联的字段 `project_name`，该字段表示 当前的 `blueprint` 唯一的属于某一张特定的 `project`
+- 一个`project`允许被多个 `blueprint` 的 `project_name` 字段指定。也就是`project`对`blueprint`是一对多关系
Review Comment:
no, it should be 1:1 relationship.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
+
+当我们创建出 `project` 对象之后：
+
+- 我们可以通过 `GET` `/project` 接口来获取具体的的某一个 `project`的信息，除了名称外，这些信息里也包含了project的更新时间和创建时间。
+- 我们可以通过 `UPDATE` `/project` 接口来更新具体的某一个 `project`的信息，比如更新其相关的 `描述` 信息。
+- 我们可以通过 `GET` `/projects` 接口来获取所有的`project`信息。
+
+2. 与此同时，我们可以通过 `GET` `/plugininfo` 接口来获取到 完整详细的 `插件` 信息，其中包括对应 `插件` 的每个表的结构信息。除此以外，我们也可以通过调用相对简单的 `GET` `/plugins` 来获取更为简洁的插件信息。简洁的插件信息仅包含插件名称，也就是`plugin_name`组成的列表，除此之外不包含其他任何信息。
+
+3. 在上述过程之后 通过 `POST` `/project_metrics` 接口来创建一组 `project` 与 `plugins` 之间的关系。使用我们前面获取到的 `project_name` 和 `plugin_name` 来构建这种关系。同时为其配置相应的 `option`.
+
+当我们创建出一组 `project_metrics` 的关系后我们就可以做更进一步的操作：
+
+- 我们可以通过 `GET` `/project_metrics` 接口来获取一组 `project_metrics` 信息，我们可以只设置 `project_name` 或只设置 `plugin_name` 来进行筛选。也可以同时设置 `project_name` 和 `plugin_name` 此时我们将获取到一个特定的 `project_metrics` 信息。
+- 我们可以通过 `UPDATE` `/project_metrics` 接口来更新特定的一组 `project_metrics` 信息。这要求我们必须同时设置好 `project_name` 和 `plugin_name`。
+
+# 关于 Project 和 Blueprint
+
+- 在 `blueprint` 的表中现在添加了一个与 `project` 相关联的字段 `project_name`，该字段表示 当前的 `blueprint` 唯一的属于某一张特定的 `project`
+- 一个`project`允许被多个 `blueprint` 的 `project_name` 字段指定。也就是`project`对`blueprint`是一对多关系
+- 可以通过以下sql 快速的查看 某个 `project` 所对应的全部 `blueprint`
+
+```
+SELECT * from `_devlake_blueprints` where `project_name`="ProjectName";
+```
+
+
+## The PluginMetric Interface
+
+
+```go
+type PluginMetric interface {
+ // returns a list of required data entities and expected features.
+ // [{ "model": "cicd_tasks", "requiredFields": {"column": "type", "execptedValue": "Deployment"}}, ...]
+ RequiredDataEntities() (data map[string]interface{},err errors.Error)
+
+ // This method returns all models of the current plugin
+ GetTablesInfo() []core.Tabler
+
+ // returns if the metric depends on Project for calculation.
+ // Currently, only dora would return true.
+ IsProjectMetric() bool
+
+ // indicates which plugins must be executed before executing this one.
+ // declare a set of dependencies with this
+ RunAfter() (PluginsNames []string,errors.Error)
+
+ // returns an empty pointer of the plugin setting struct.
+ // (no concrete usage at this point)
+ Settings() (p interface{})
+}
+```
+
+## models
+
+The following is the table structure data of `project` related.
+
+```go
+type Projects struct {
+ Name string `gorm:"primaryKey" gorm:"type:varchar(255)"`
+ describe string `gorm:"type:text"`
+ CreatedAt time.time
+ UpdatedAt time.time
+}
+
+func (Projects) TableName() {
+ return "_devlake_projects"
+}
+
+type ProjectMetrics struct {
+ ProjectName string `gorm:"primaryKey" gorm:"type:varchar(255)"`
+ PluginName string `gorm:"primaryKey" gorm:"type:varchar(255)"`
+ PluginOption string `gorm:"type:text"`
+}
+
+func (ProjectMetrics) TableName() {
+ return "_devlake_project_metrics"
+}
+
+type ProjectMetrics struct {
+ ProjectName string `gorm:"primaryKey" gorm:"type:varchar(255)"`
+ PluginName string `gorm:"primaryKey" gorm:"type:varchar(255)"`
+ PluginOption string `gorm:"type:text"`
+}
+
+type ProjectMapping struct {
+ ProjectName string `gorm:"primaryKey;type:varchar(255)"`
+ ScopeID string `gorm:"primaryKey;type:varchar(255)"`
+}
+
+func (ProjectMapping) TableName() string {
+ return "_devlake_project_mapping"
+}
+
+```
+
+## api
Review Comment:
no need to put it here, api doc should be rendered in swag doc.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
+
+当我们创建出 `project` 对象之后：
+
+- 我们可以通过 `GET` `/project` 接口来获取具体的的某一个 `project`的信息，除了名称外，这些信息里也包含了project的更新时间和创建时间。
+- 我们可以通过 `UPDATE` `/project` 接口来更新具体的某一个 `project`的信息，比如更新其相关的 `描述` 信息。
+- 我们可以通过 `GET` `/projects` 接口来获取所有的`project`信息。
Review Comment:
should support `search` and `pagination` as well
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
+
+当我们创建出 `project` 对象之后：
+
+- 我们可以通过 `GET` `/project` 接口来获取具体的的某一个 `project`的信息，除了名称外，这些信息里也包含了project的更新时间和创建时间。
+- 我们可以通过 `UPDATE` `/project` 接口来更新具体的某一个 `project`的信息，比如更新其相关的 `描述` 信息。
+- 我们可以通过 `GET` `/projects` 接口来获取所有的`project`信息。
+
+2. 与此同时，我们可以通过 `GET` `/plugininfo` 接口来获取到 完整详细的 `插件` 信息，其中包括对应 `插件` 的每个表的结构信息。除此以外，我们也可以通过调用相对简单的 `GET` `/plugins` 来获取更为简洁的插件信息。简洁的插件信息仅包含插件名称，也就是`plugin_name`组成的列表，除此之外不包含其他任何信息。
Review Comment:
"为了能给 project 指定需要的指标，你可以通过 `GET /plugins` 接口来获取插件信息。"
复杂的接口就不用提了，那个主要返回 model 信息对这个场景帮助不大。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] mappjzc commented on a diff in pull request #308: WIP Project framework
Posted by GitBox <gi...@apache.org>.
mappjzc commented on code in PR #308:
URL: https://github.com/apache/incubator-devlake-website/pull/308#discussion_r1011625533
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,226 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+---
+
+## Summary
+`Project` is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like `DORA`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | gitlab | {} | true |
+| project_2 | gitlab | {} | false |
+| project_2 | github | {} | true |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `scope_id` | bigint | 255 | the scope id by project mapping | PK |
+
+
+| **project_name** | **scope_id** |
+| ---------------- | ------------ |
+| project_1 | 1 |
+| project_1 | 2 |
+| project_2 | 1 |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `POST` `/project` 接口来创建一个全新的 `project` 对象，该对象需要一个 `project_name`字段，作为其名称，该名称唯一。
+
+当我们创建出 `project` 对象之后：
+
+- 我们可以通过 `GET` `/project` 接口来获取具体的的某一个 `project`的信息，除了名称外，这些信息里也包含了project的更新时间和创建时间。
Review Comment:
we have define /projects api
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] klesh commented on a diff in pull request #308: WIP Project framework
Posted by GitBox <gi...@apache.org>.
klesh commented on code in PR #308:
URL: https://github.com/apache/incubator-devlake-website/pull/308#discussion_r1034494828
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
Review Comment:
For some metric calculations such as the DORA metric,
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like `Dora`, we need to correctly associate different [Scope](../Glossary.md#data-scope) under different data sources through a method.
+
+So, we created `Project`
Review Comment:
delete
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
Review Comment:
For example, we may use Gitlab for code version control, Jenkins for CI/CD, to calculate PR deployment cycle time, we need to know which Gitlab Projects and Jenkins Jobs are related for correctness and performance reasons.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like `Dora`, we need to correctly associate different [Scope](../Glossary.md#data-scope) under different data sources through a method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate `project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
Review Comment:
I think it would be better to rephrase Line24-31 into a section named "Interfaces" and it should be placed after the "Model" section.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like `Dora`, we need to correctly associate different [Scope](../Glossary.md#data-scope) under different data sources through a method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate `project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
+
+By visiting `[Your Config-UI Host]/api/swagger/index.html`, you can go to the swagger document of `Devlake` deployed. In this document, `framework/projects` `framework/ProjectMetrics`, and `framework/plugins ` is the related API of `project`.
+
+To protect the integrity of the data in the database as much as possible, we recommend using the API to perform corresponding operations instead of directly using SQL statements to operate the tables in the database.
+
+When we need to manipulate the `projects` table, we can use the `framework/projects`:
+- We can create a new `project` object through the `POST` `/projects` interface, which requires a `projectName` field as its name, which is unique.
+
+After we created the `project` object or we already had a `project`:
+- We can use the `GET /projects/:projectName` interface to obtain specific `project` information. In addition to the name, this information also includes the update time and creation time of the project, and will be associated with all The relevant `project_metrics` and `project_mapping` information of the current `project` will be returned after integration.
+- We can use the `PATCH /projects/:projectName` interface to update the information of a specific `project`, such as updating its related `description` information.
+- We can get all `project` information through the `GET /projects` interface. The returned information can be filtered by setting the parameter `search`, and the returned information can be paged by setting the parameters `page`, and `pageSize`.
+
+After we created the `project` object or we already had a `project`:
+- we can use the `POST /project_metrics/:projectName/:pluginName` to create a set of relationships between `project` and `Metrics Plugin`, Use the `projectName` and `pluginName` we obtained earlier to build this relationship. At the same time configure the corresponding `option` for it.
Review Comment:
it should be `/projects/:projectName/metrics`, we should update the code as well.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like `Dora`, we need to correctly associate different [Scope](../Glossary.md#data-scope) under different data sources through a method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate `project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
+
+By visiting `[Your Config-UI Host]/api/swagger/index.html`, you can go to the swagger document of `Devlake` deployed. In this document, `framework/projects` `framework/ProjectMetrics`, and `framework/plugins ` is the related API of `project`.
+
+To protect the integrity of the data in the database as much as possible, we recommend using the API to perform corresponding operations instead of directly using SQL statements to operate the tables in the database.
+
+When we need to manipulate the `projects` table, we can use the `framework/projects`:
+- We can create a new `project` object through the `POST` `/projects` interface, which requires a `projectName` field as its name, which is unique.
+
+After we created the `project` object or we already had a `project`:
+- We can use the `GET /projects/:projectName` interface to obtain specific `project` information. In addition to the name, this information also includes the update time and creation time of the project, and will be associated with all The relevant `project_metrics` and `project_mapping` information of the current `project` will be returned after integration.
+- We can use the `PATCH /projects/:projectName` interface to update the information of a specific `project`, such as updating its related `description` information.
+- We can get all `project` information through the `GET /projects` interface. The returned information can be filtered by setting the parameter `search`, and the returned information can be paged by setting the parameters `page`, and `pageSize`.
+
+After we created the `project` object or we already had a `project`:
+- we can use the `POST /project_metrics/:projectName/:pluginName` to create a set of relationships between `project` and `Metrics Plugin`, Use the `projectName` and `pluginName` we obtained earlier to build this relationship. At the same time configure the corresponding `option` for it.
+
+After we create a `project_metrics` relation or we already had one:
+- We can use the `GET /project_metrics/:projectName/:pluginName` to get a set of `project_metrics` information.
+- We can use the `PATCH /project_metrics/:projectName/:pluginName` to update the `project_metrics` information.
+
+If we need to know which plugins are `Metrics Plugin`:
+- we can use the `GET` `/plugins`, it will return a list of all plugin names and information about the plugins and `Metrics Plugin`.
+
+# The interface that needs to be implemented
+
+We divide plugins into three categories
+- The first category is `Base Plugin`, such as `core`, `helper`, etc. These plug-ins provide basic support services for other plug-ins.
Review Comment:
no such thing as `Base Plugin`. `core` and `helper` are not plugins.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
Review Comment:
However, in most cases, we have multiple Gitlab Projects / Jenkins Jobs that belong to different teams/agendas in our Apache DevLake database.
To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a project consists of a set of [Scopes](../Glossary.md#data-scope), i.e., a couple of Gitlab Projects, Jira Boards or Jenkins Jobs, etc.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like `Dora`, we need to correctly associate different [Scope](../Glossary.md#data-scope) under different data sources through a method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate `project` with `blueprint` one-to-one.
Review Comment:
change "one-to-one" to "in one-to-one manner"
this sentence is describe the relationship between `project` and `blueprint`, which should be moved to the Models section.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like `Dora`, we need to correctly associate different [Scope](../Glossary.md#data-scope) under different data sources through a method.
Review Comment:
delete
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like `Dora`, we need to correctly associate different [Scope](../Glossary.md#data-scope) under different data sources through a method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate `project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
Review Comment:
rename suggestion: "How to manage project via API"
and I don't think we need to be so specific (line 87~107)
Just tell developers to check the swagger doc(with a link) and lay out the major endpoints would be good enough:
For API specification, please check the [swagger doc](how-to-view-the-swagger-doc-link)
Related endpoints:
1. `/projects`
2. `/projects/:projectName/metrics`
3. `/plugins`
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like `Dora`, we need to correctly associate different [Scope](../Glossary.md#data-scope) under different data sources through a method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate `project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
+
+By visiting `[Your Config-UI Host]/api/swagger/index.html`, you can go to the swagger document of `Devlake` deployed. In this document, `framework/projects` `framework/ProjectMetrics`, and `framework/plugins ` is the related API of `project`.
+
+To protect the integrity of the data in the database as much as possible, we recommend using the API to perform corresponding operations instead of directly using SQL statements to operate the tables in the database.
+
+When we need to manipulate the `projects` table, we can use the `framework/projects`:
+- We can create a new `project` object through the `POST` `/projects` interface, which requires a `projectName` field as its name, which is unique.
+
+After we created the `project` object or we already had a `project`:
+- We can use the `GET /projects/:projectName` interface to obtain specific `project` information. In addition to the name, this information also includes the update time and creation time of the project, and will be associated with all The relevant `project_metrics` and `project_mapping` information of the current `project` will be returned after integration.
+- We can use the `PATCH /projects/:projectName` interface to update the information of a specific `project`, such as updating its related `description` information.
+- We can get all `project` information through the `GET /projects` interface. The returned information can be filtered by setting the parameter `search`, and the returned information can be paged by setting the parameters `page`, and `pageSize`.
+
+After we created the `project` object or we already had a `project`:
+- we can use the `POST /project_metrics/:projectName/:pluginName` to create a set of relationships between `project` and `Metrics Plugin`, Use the `projectName` and `pluginName` we obtained earlier to build this relationship. At the same time configure the corresponding `option` for it.
+
+After we create a `project_metrics` relation or we already had one:
+- We can use the `GET /project_metrics/:projectName/:pluginName` to get a set of `project_metrics` information.
+- We can use the `PATCH /project_metrics/:projectName/:pluginName` to update the `project_metrics` information.
+
+If we need to know which plugins are `Metrics Plugin`:
+- we can use the `GET` `/plugins`, it will return a list of all plugin names and information about the plugins and `Metrics Plugin`.
+
+# The interface that needs to be implemented
+
+We divide plugins into three categories
+- The first category is `Base Plugin`, such as `core`, `helper`, etc. These plug-ins provide basic support services for other plug-ins.
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
+
+The second and third types of plug-ins here need to implement the corresponding interface to support the project.
+
+## Data Source Plugin
+
+`Data Source Plugin` is a plugin used to extract data from a data source.
+
+For example `GitLab` `GitHub` `Jira` `Jenkins` etc.
+
+These plugins, from various data sources, extract data into the database and store them, they deal directly with the data source, so we classify them as `Data Source Plugin`.
+
+## the DataSourcePluginBlueprintV200 interface
Review Comment:
Move to the "Interfaces" section as well.
No need to be so detailed, laying out so much code is confusing.
We describe the big picture here and ask the developer to check the source code for detail, there are comments laying around inside the source file.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like `Dora`, we need to correctly associate different [Scope](../Glossary.md#data-scope) under different data sources through a method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate `project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
+
+By visiting `[Your Config-UI Host]/api/swagger/index.html`, you can go to the swagger document of `Devlake` deployed. In this document, `framework/projects` `framework/ProjectMetrics`, and `framework/plugins ` is the related API of `project`.
+
+To protect the integrity of the data in the database as much as possible, we recommend using the API to perform corresponding operations instead of directly using SQL statements to operate the tables in the database.
+
+When we need to manipulate the `projects` table, we can use the `framework/projects`:
+- We can create a new `project` object through the `POST` `/projects` interface, which requires a `projectName` field as its name, which is unique.
+
+After we created the `project` object or we already had a `project`:
+- We can use the `GET /projects/:projectName` interface to obtain specific `project` information. In addition to the name, this information also includes the update time and creation time of the project, and will be associated with all The relevant `project_metrics` and `project_mapping` information of the current `project` will be returned after integration.
+- We can use the `PATCH /projects/:projectName` interface to update the information of a specific `project`, such as updating its related `description` information.
+- We can get all `project` information through the `GET /projects` interface. The returned information can be filtered by setting the parameter `search`, and the returned information can be paged by setting the parameters `page`, and `pageSize`.
+
+After we created the `project` object or we already had a `project`:
+- we can use the `POST /project_metrics/:projectName/:pluginName` to create a set of relationships between `project` and `Metrics Plugin`, Use the `projectName` and `pluginName` we obtained earlier to build this relationship. At the same time configure the corresponding `option` for it.
+
+After we create a `project_metrics` relation or we already had one:
+- We can use the `GET /project_metrics/:projectName/:pluginName` to get a set of `project_metrics` information.
+- We can use the `PATCH /project_metrics/:projectName/:pluginName` to update the `project_metrics` information.
+
+If we need to know which plugins are `Metrics Plugin`:
+- we can use the `GET` `/plugins`, it will return a list of all plugin names and information about the plugins and `Metrics Plugin`.
+
+# The interface that needs to be implemented
+
+We divide plugins into three categories
+- The first category is `Base Plugin`, such as `core`, `helper`, etc. These plug-ins provide basic support services for other plug-ins.
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
Review Comment:
Combine this information(Line133-134) to the "Interface" section
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] Startrekzky commented on a diff in pull request #308: WIP Project framework
Posted by GitBox <gi...@apache.org>.
Startrekzky commented on code in PR #308:
URL: https://github.com/apache/incubator-devlake-website/pull/308#discussion_r1009017443
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,152 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ Project is an entity object that can be used for Dora to associate pr to deployment/issue.
+---
+
+## Summary
+Project is an entity object that can be used for Dora to associate pr to deployment/issue.
+
+It contains the following two models:
Review Comment:
You can add two tables to make the data models clearer.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,152 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ Project is an entity object that can be used for Dora to associate pr to deployment/issue.
Review Comment:
`Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,152 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ Project is an entity object that can be used for Dora to associate pr to deployment/issue.
+---
+
+## Summary
+Project is an entity object that can be used for Dora to associate pr to deployment/issue.
Review Comment:
The same as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] Startrekzky merged pull request #308: Project framework
Posted by GitBox <gi...@apache.org>.
Startrekzky merged PR #308:
URL: https://github.com/apache/incubator-devlake-website/pull/308
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] klesh commented on a diff in pull request #308: Project framework
Posted by GitBox <gi...@apache.org>.
klesh commented on code in PR #308:
URL: https://github.com/apache/incubator-devlake-website/pull/308#discussion_r1057099894
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+For some metric calculations such as the `DORA` metric, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we may use `Gitlab` for code version control, `Jenkins` for CI/CD, to calculate PR deployment cycle time, we need to know which `Gitlab Projects` and `Jenkins Jobs` are related for correctness and performance reasons.
+
+However, in most cases, we have multiple `Gitlab Projects` / `Jenkins Jobs` that belong to different teams/agendas in our Apache DevLake database.
+
+To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a `project` consists of a set of [Scopes](../Overview/KeyConcepts.md#data-scope), i.e., a couple of `Gitlab Projects`, `Jira Boards` or `Jenkins Jobs`, etc.
+
+`Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metric_settings` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric_settings
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Overview/KeyConcepts.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Overview/KeyConcepts.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+## _devlake_blueprint
+
+By adding the `project_name` field to the `blueprint` Models, we associate `project` with `blueprint` in one-to-one manner.
+# How to manage project via API
+
+For API specification, please check the swagger doc(by visiting `[Your Config-UI Host]/api/swagger/index.html`).
+Related endpoints:
+
+1. /projects
+2. /projects/:projectName/metrics
+3. /plugins
+
+# The interface that needs to be implemented
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
Review Comment:
I feel this paragraph is not necessary. Viewers would determine whether they should learn it or not.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+For some metric calculations such as the `DORA` metric, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we may use `Gitlab` for code version control, `Jenkins` for CI/CD, to calculate PR deployment cycle time, we need to know which `Gitlab Projects` and `Jenkins Jobs` are related for correctness and performance reasons.
+
+However, in most cases, we have multiple `Gitlab Projects` / `Jenkins Jobs` that belong to different teams/agendas in our Apache DevLake database.
+
+To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a `project` consists of a set of [Scopes](../Overview/KeyConcepts.md#data-scope), i.e., a couple of `Gitlab Projects`, `Jira Boards` or `Jenkins Jobs`, etc.
+
+`Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metric_settings` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric_settings
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Overview/KeyConcepts.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Overview/KeyConcepts.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+## _devlake_blueprint
+
+By adding the `project_name` field to the `blueprint` Models, we associate `project` with `blueprint` in one-to-one manner.
+# How to manage project via API
+
+For API specification, please check the swagger doc(by visiting `[Your Config-UI Host]/api/swagger/index.html`).
+Related endpoints:
+
+1. /projects
+2. /projects/:projectName/metrics
+3. /plugins
+
+# The interface that needs to be implemented
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+We divide plugins into three categories
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
Review Comment:
what?
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+For some metric calculations such as the `DORA` metric, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we may use `Gitlab` for code version control, `Jenkins` for CI/CD, to calculate PR deployment cycle time, we need to know which `Gitlab Projects` and `Jenkins Jobs` are related for correctness and performance reasons.
+
+However, in most cases, we have multiple `Gitlab Projects` / `Jenkins Jobs` that belong to different teams/agendas in our Apache DevLake database.
+
+To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a `project` consists of a set of [Scopes](../Overview/KeyConcepts.md#data-scope), i.e., a couple of `Gitlab Projects`, `Jira Boards` or `Jenkins Jobs`, etc.
+
+`Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metric_settings` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric_settings
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Overview/KeyConcepts.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Overview/KeyConcepts.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+## _devlake_blueprint
+
+By adding the `project_name` field to the `blueprint` Models, we associate `project` with `blueprint` in one-to-one manner.
+# How to manage project via API
+
+For API specification, please check the swagger doc(by visiting `[Your Config-UI Host]/api/swagger/index.html`).
+Related endpoints:
+
+1. /projects
+2. /projects/:projectName/metrics
+3. /plugins
+
+# The interface that needs to be implemented
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+We divide plugins into three categories
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
+
+The second and third types of plug-ins here need to implement the corresponding interface to support the project.
+
+## Data Source Plugin
+
+`Data Source Plugin` is a plugin used to extract data from a data source.
+
+For example `GitLab` `GitHub` `Jira` `Jenkins` etc.
+
+These plugins, from various data sources, extract data into the database and store them, they deal directly with the data source, so we classify them as `Data Source Plugin`.
+
+## the DataSourcePluginBlueprintV200 interface
+
+`Data Source Plugin` needs to implement `DataSourcePluginBlueprintV200` interface to support `project`
+
+The interface definition for this interface is as follows
+
+```go
+// DataSourcePluginBlueprintV200 extends the V100 to provide support for
+// Project, so that complex metrics like DORA can be implemented based on a set
+// of Data Scopes
+type DataSourcePluginBlueprintV200 interface {
+ MakeDataSourcePipelinePlanV200(
+ connectionId uint64,
+ scopes []*BlueprintScopeV200,
+ syncPolicy BlueprintSyncPolicy,
+ ) (PipelinePlan, []Scope, errors.Error)
+}
+```
+
+`project` needs to provide a specific set of [Scopes](../Overview/KeyConcepts.md#data-scope) for a specific `connection` to the plug-in through this interface, and then obtain the plug-in involved in the `PipelineTask` All `plugins` and corresponding parameter information. At the same time, the plug-in needs to convert entities like `repo` and `board` in the data source into a `scope interface` that `project` can understand
Review Comment:
The `DataSourcePluginBlueprintV200` is introduced for the plugin to tell the framework how should the pipeline to be executed and what specific `Domain Layer Scopes` would be produced based on users input (`connectionId`, `Tool Layer Scopes` and `SyncPolicy`)
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+For some metric calculations such as the `DORA` metric, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we may use `Gitlab` for code version control, `Jenkins` for CI/CD, to calculate PR deployment cycle time, we need to know which `Gitlab Projects` and `Jenkins Jobs` are related for correctness and performance reasons.
+
+However, in most cases, we have multiple `Gitlab Projects` / `Jenkins Jobs` that belong to different teams/agendas in our Apache DevLake database.
+
+To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a `project` consists of a set of [Scopes](../Overview/KeyConcepts.md#data-scope), i.e., a couple of `Gitlab Projects`, `Jira Boards` or `Jenkins Jobs`, etc.
+
+`Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metric_settings` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric_settings
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Overview/KeyConcepts.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Overview/KeyConcepts.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+## _devlake_blueprint
+
+By adding the `project_name` field to the `blueprint` Models, we associate `project` with `blueprint` in one-to-one manner.
+# How to manage project via API
+
+For API specification, please check the swagger doc(by visiting `[Your Config-UI Host]/api/swagger/index.html`).
+Related endpoints:
+
+1. /projects
+2. /projects/:projectName/metrics
+3. /plugins
+
+# The interface that needs to be implemented
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+We divide plugins into three categories
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
+
+The second and third types of plug-ins here need to implement the corresponding interface to support the project.
+
+## Data Source Plugin
+
+`Data Source Plugin` is a plugin used to extract data from a data source.
+
+For example `GitLab` `GitHub` `Jira` `Jenkins` etc.
+
+These plugins, from various data sources, extract data into the database and store them, they deal directly with the data source, so we classify them as `Data Source Plugin`.
+
+## the DataSourcePluginBlueprintV200 interface
+
+`Data Source Plugin` needs to implement `DataSourcePluginBlueprintV200` interface to support `project`
+
+The interface definition for this interface is as follows
+
+```go
+// DataSourcePluginBlueprintV200 extends the V100 to provide support for
+// Project, so that complex metrics like DORA can be implemented based on a set
+// of Data Scopes
+type DataSourcePluginBlueprintV200 interface {
+ MakeDataSourcePipelinePlanV200(
+ connectionId uint64,
+ scopes []*BlueprintScopeV200,
+ syncPolicy BlueprintSyncPolicy,
+ ) (PipelinePlan, []Scope, errors.Error)
+}
+```
+
+`project` needs to provide a specific set of [Scopes](../Overview/KeyConcepts.md#data-scope) for a specific `connection` to the plug-in through this interface, and then obtain the plug-in involved in the `PipelineTask` All `plugins` and corresponding parameter information. At the same time, the plug-in needs to convert entities like `repo` and `board` in the data source into a `scope interface` that `project` can understand
+
+The corresponding `scope interface` has been implemented at following files of in the framework layer:
+- `models/domainlayer/ticket/board.go`
+- `models/domainlayer/code/repo.go`
+
+In the `plugins/gitlab/impl/impl.go` file, there is a `Gitlab` plugin implementation of the above interface, which can be used as a sample reference.
+
+And the `plugins/gitlab/api/blueprint_v200.go` contains implementation details.
+
+The following files contain the models that the relevant implementations depend on for reference:
+- `plugins/gitlab/models/project.go`
+- `plugins/gitlab/models/transformation_rule.go`
+
+## Metrics Plugin
+The `Metrics Plugin` is based on the data collected by the `Data Source Plugin` to do secondary calculations.
+
+For example `Dora`, and `Refdff` plugins belong to the `Metrics Plugin`
Review Comment:
already mentioned above, do not repeat.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+For some metric calculations such as the `DORA` metric, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we may use `Gitlab` for code version control, `Jenkins` for CI/CD, to calculate PR deployment cycle time, we need to know which `Gitlab Projects` and `Jenkins Jobs` are related for correctness and performance reasons.
+
+However, in most cases, we have multiple `Gitlab Projects` / `Jenkins Jobs` that belong to different teams/agendas in our Apache DevLake database.
+
+To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a `project` consists of a set of [Scopes](../Overview/KeyConcepts.md#data-scope), i.e., a couple of `Gitlab Projects`, `Jira Boards` or `Jenkins Jobs`, etc.
+
+`Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metric_settings` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric_settings
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Overview/KeyConcepts.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Overview/KeyConcepts.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+## _devlake_blueprint
+
+By adding the `project_name` field to the `blueprint` Models, we associate `project` with `blueprint` in one-to-one manner.
+# How to manage project via API
+
+For API specification, please check the swagger doc(by visiting `[Your Config-UI Host]/api/swagger/index.html`).
+Related endpoints:
+
+1. /projects
+2. /projects/:projectName/metrics
+3. /plugins
+
+# The interface that needs to be implemented
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+We divide plugins into three categories
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
+
+The second and third types of plug-ins here need to implement the corresponding interface to support the project.
+
+## Data Source Plugin
+
+`Data Source Plugin` is a plugin used to extract data from a data source.
+
+For example `GitLab` `GitHub` `Jira` `Jenkins` etc.
+
+These plugins, from various data sources, extract data into the database and store them, they deal directly with the data source, so we classify them as `Data Source Plugin`.
+
+## the DataSourcePluginBlueprintV200 interface
+
+`Data Source Plugin` needs to implement `DataSourcePluginBlueprintV200` interface to support `project`
+
+The interface definition for this interface is as follows
+
+```go
+// DataSourcePluginBlueprintV200 extends the V100 to provide support for
+// Project, so that complex metrics like DORA can be implemented based on a set
+// of Data Scopes
+type DataSourcePluginBlueprintV200 interface {
+ MakeDataSourcePipelinePlanV200(
+ connectionId uint64,
+ scopes []*BlueprintScopeV200,
+ syncPolicy BlueprintSyncPolicy,
+ ) (PipelinePlan, []Scope, errors.Error)
+}
+```
+
+`project` needs to provide a specific set of [Scopes](../Overview/KeyConcepts.md#data-scope) for a specific `connection` to the plug-in through this interface, and then obtain the plug-in involved in the `PipelineTask` All `plugins` and corresponding parameter information. At the same time, the plug-in needs to convert entities like `repo` and `board` in the data source into a `scope interface` that `project` can understand
+
+The corresponding `scope interface` has been implemented at following files of in the framework layer:
+- `models/domainlayer/ticket/board.go`
+- `models/domainlayer/code/repo.go`
+
+In the `plugins/gitlab/impl/impl.go` file, there is a `Gitlab` plugin implementation of the above interface, which can be used as a sample reference.
+
+And the `plugins/gitlab/api/blueprint_v200.go` contains implementation details.
+
+The following files contain the models that the relevant implementations depend on for reference:
+- `plugins/gitlab/models/project.go`
+- `plugins/gitlab/models/transformation_rule.go`
+
+## Metrics Plugin
+The `Metrics Plugin` is based on the data collected by the `Data Source Plugin` to do secondary calculations.
Review Comment:
already mentioned above, do not repeat.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+For some metric calculations such as the `DORA` metric, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we may use `Gitlab` for code version control, `Jenkins` for CI/CD, to calculate PR deployment cycle time, we need to know which `Gitlab Projects` and `Jenkins Jobs` are related for correctness and performance reasons.
+
+However, in most cases, we have multiple `Gitlab Projects` / `Jenkins Jobs` that belong to different teams/agendas in our Apache DevLake database.
+
+To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a `project` consists of a set of [Scopes](../Overview/KeyConcepts.md#data-scope), i.e., a couple of `Gitlab Projects`, `Jira Boards` or `Jenkins Jobs`, etc.
+
+`Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metric_settings` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric_settings
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Overview/KeyConcepts.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Overview/KeyConcepts.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+## _devlake_blueprint
+
+By adding the `project_name` field to the `blueprint` Models, we associate `project` with `blueprint` in one-to-one manner.
+# How to manage project via API
+
+For API specification, please check the swagger doc(by visiting `[Your Config-UI Host]/api/swagger/index.html`).
+Related endpoints:
+
+1. /projects
+2. /projects/:projectName/metrics
+3. /plugins
+
+# The interface that needs to be implemented
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+We divide plugins into three categories
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
+
+The second and third types of plug-ins here need to implement the corresponding interface to support the project.
+
+## Data Source Plugin
+
+`Data Source Plugin` is a plugin used to extract data from a data source.
Review Comment:
It is mentioned in line 96 already. no need to repeat
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+For some metric calculations such as the `DORA` metric, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we may use `Gitlab` for code version control, `Jenkins` for CI/CD, to calculate PR deployment cycle time, we need to know which `Gitlab Projects` and `Jenkins Jobs` are related for correctness and performance reasons.
+
+However, in most cases, we have multiple `Gitlab Projects` / `Jenkins Jobs` that belong to different teams/agendas in our Apache DevLake database.
+
+To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a `project` consists of a set of [Scopes](../Overview/KeyConcepts.md#data-scope), i.e., a couple of `Gitlab Projects`, `Jira Boards` or `Jenkins Jobs`, etc.
+
+`Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metric_settings` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric_settings
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Overview/KeyConcepts.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Overview/KeyConcepts.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+## _devlake_blueprint
+
+By adding the `project_name` field to the `blueprint` Models, we associate `project` with `blueprint` in one-to-one manner.
+# How to manage project via API
+
+For API specification, please check the swagger doc(by visiting `[Your Config-UI Host]/api/swagger/index.html`).
+Related endpoints:
+
+1. /projects
+2. /projects/:projectName/metrics
+3. /plugins
+
+# The interface that needs to be implemented
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+We divide plugins into three categories
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
Review Comment:
where is the first category?
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+For some metric calculations such as the `DORA` metric, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we may use `Gitlab` for code version control, `Jenkins` for CI/CD, to calculate PR deployment cycle time, we need to know which `Gitlab Projects` and `Jenkins Jobs` are related for correctness and performance reasons.
+
+However, in most cases, we have multiple `Gitlab Projects` / `Jenkins Jobs` that belong to different teams/agendas in our Apache DevLake database.
+
+To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a `project` consists of a set of [Scopes](../Overview/KeyConcepts.md#data-scope), i.e., a couple of `Gitlab Projects`, `Jira Boards` or `Jenkins Jobs`, etc.
+
+`Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metric_settings` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric_settings
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Overview/KeyConcepts.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Overview/KeyConcepts.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+## _devlake_blueprint
+
+By adding the `project_name` field to the `blueprint` Models, we associate `project` with `blueprint` in one-to-one manner.
+# How to manage project via API
+
+For API specification, please check the swagger doc(by visiting `[Your Config-UI Host]/api/swagger/index.html`).
+Related endpoints:
+
+1. /projects
+2. /projects/:projectName/metrics
+3. /plugins
+
+# The interface that needs to be implemented
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+We divide plugins into three categories
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
+
+The second and third types of plug-ins here need to implement the corresponding interface to support the project.
+
+## Data Source Plugin
+
+`Data Source Plugin` is a plugin used to extract data from a data source.
+
+For example `GitLab` `GitHub` `Jira` `Jenkins` etc.
+
+These plugins, from various data sources, extract data into the database and store them, they deal directly with the data source, so we classify them as `Data Source Plugin`.
+
+## the DataSourcePluginBlueprintV200 interface
+
+`Data Source Plugin` needs to implement `DataSourcePluginBlueprintV200` interface to support `project`
+
+The interface definition for this interface is as follows
+
+```go
+// DataSourcePluginBlueprintV200 extends the V100 to provide support for
+// Project, so that complex metrics like DORA can be implemented based on a set
+// of Data Scopes
+type DataSourcePluginBlueprintV200 interface {
+ MakeDataSourcePipelinePlanV200(
+ connectionId uint64,
+ scopes []*BlueprintScopeV200,
+ syncPolicy BlueprintSyncPolicy,
+ ) (PipelinePlan, []Scope, errors.Error)
+}
+```
+
+`project` needs to provide a specific set of [Scopes](../Overview/KeyConcepts.md#data-scope) for a specific `connection` to the plug-in through this interface, and then obtain the plug-in involved in the `PipelineTask` All `plugins` and corresponding parameter information. At the same time, the plug-in needs to convert entities like `repo` and `board` in the data source into a `scope interface` that `project` can understand
+
+The corresponding `scope interface` has been implemented at following files of in the framework layer:
+- `models/domainlayer/ticket/board.go`
+- `models/domainlayer/code/repo.go`
Review Comment:
cicd_scope is missing
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+For some metric calculations such as the `DORA` metric, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we may use `Gitlab` for code version control, `Jenkins` for CI/CD, to calculate PR deployment cycle time, we need to know which `Gitlab Projects` and `Jenkins Jobs` are related for correctness and performance reasons.
+
+However, in most cases, we have multiple `Gitlab Projects` / `Jenkins Jobs` that belong to different teams/agendas in our Apache DevLake database.
+
+To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a `project` consists of a set of [Scopes](../Overview/KeyConcepts.md#data-scope), i.e., a couple of `Gitlab Projects`, `Jira Boards` or `Jenkins Jobs`, etc.
+
+`Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metric_settings` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric_settings
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Overview/KeyConcepts.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Overview/KeyConcepts.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+## _devlake_blueprint
+
+By adding the `project_name` field to the `blueprint` Models, we associate `project` with `blueprint` in one-to-one manner.
+# How to manage project via API
+
+For API specification, please check the swagger doc(by visiting `[Your Config-UI Host]/api/swagger/index.html`).
+Related endpoints:
+
+1. /projects
+2. /projects/:projectName/metrics
+3. /plugins
+
+# The interface that needs to be implemented
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+We divide plugins into three categories
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
+
+The second and third types of plug-ins here need to implement the corresponding interface to support the project.
+
+## Data Source Plugin
+
+`Data Source Plugin` is a plugin used to extract data from a data source.
+
+For example `GitLab` `GitHub` `Jira` `Jenkins` etc.
+
+These plugins, from various data sources, extract data into the database and store them, they deal directly with the data source, so we classify them as `Data Source Plugin`.
+
+## the DataSourcePluginBlueprintV200 interface
+
+`Data Source Plugin` needs to implement `DataSourcePluginBlueprintV200` interface to support `project`
+
+The interface definition for this interface is as follows
+
+```go
+// DataSourcePluginBlueprintV200 extends the V100 to provide support for
+// Project, so that complex metrics like DORA can be implemented based on a set
+// of Data Scopes
+type DataSourcePluginBlueprintV200 interface {
+ MakeDataSourcePipelinePlanV200(
+ connectionId uint64,
+ scopes []*BlueprintScopeV200,
+ syncPolicy BlueprintSyncPolicy,
+ ) (PipelinePlan, []Scope, errors.Error)
+}
+```
+
+`project` needs to provide a specific set of [Scopes](../Overview/KeyConcepts.md#data-scope) for a specific `connection` to the plug-in through this interface, and then obtain the plug-in involved in the `PipelineTask` All `plugins` and corresponding parameter information. At the same time, the plug-in needs to convert entities like `repo` and `board` in the data source into a `scope interface` that `project` can understand
+
+The corresponding `scope interface` has been implemented at following files of in the framework layer:
+- `models/domainlayer/ticket/board.go`
+- `models/domainlayer/code/repo.go`
+
+In the `plugins/gitlab/impl/impl.go` file, there is a `Gitlab` plugin implementation of the above interface, which can be used as a sample reference.
Review Comment:
remove **sample**
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+For some metric calculations such as the `DORA` metric, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we may use `Gitlab` for code version control, `Jenkins` for CI/CD, to calculate PR deployment cycle time, we need to know which `Gitlab Projects` and `Jenkins Jobs` are related for correctness and performance reasons.
+
+However, in most cases, we have multiple `Gitlab Projects` / `Jenkins Jobs` that belong to different teams/agendas in our Apache DevLake database.
+
+To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a `project` consists of a set of [Scopes](../Overview/KeyConcepts.md#data-scope), i.e., a couple of `Gitlab Projects`, `Jira Boards` or `Jenkins Jobs`, etc.
+
+`Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metric_settings` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric_settings
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Overview/KeyConcepts.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Overview/KeyConcepts.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+## _devlake_blueprint
+
+By adding the `project_name` field to the `blueprint` Models, we associate `project` with `blueprint` in one-to-one manner.
+# How to manage project via API
+
+For API specification, please check the swagger doc(by visiting `[Your Config-UI Host]/api/swagger/index.html`).
+Related endpoints:
+
+1. /projects
+2. /projects/:projectName/metrics
+3. /plugins
+
+# The interface that needs to be implemented
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+We divide plugins into three categories
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
+
+The second and third types of plug-ins here need to implement the corresponding interface to support the project.
+
+## Data Source Plugin
+
+`Data Source Plugin` is a plugin used to extract data from a data source.
+
+For example `GitLab` `GitHub` `Jira` `Jenkins` etc.
+
+These plugins, from various data sources, extract data into the database and store them, they deal directly with the data source, so we classify them as `Data Source Plugin`.
+
+## the DataSourcePluginBlueprintV200 interface
+
+`Data Source Plugin` needs to implement `DataSourcePluginBlueprintV200` interface to support `project`
Review Comment:
same as above
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,184 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+For some metric calculations such as the `DORA` metric, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we may use `Gitlab` for code version control, `Jenkins` for CI/CD, to calculate PR deployment cycle time, we need to know which `Gitlab Projects` and `Jenkins Jobs` are related for correctness and performance reasons.
+
+However, in most cases, we have multiple `Gitlab Projects` / `Jenkins Jobs` that belong to different teams/agendas in our Apache DevLake database.
+
+To distinguish them into different groups. The `Project` is introduced in v0.15. Essentially, a `project` consists of a set of [Scopes](../Overview/KeyConcepts.md#data-scope), i.e., a couple of `Gitlab Projects`, `Jira Boards` or `Jenkins Jobs`, etc.
+
+`Project` is **a set of [Scope](../Overview/KeyConcepts.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metric_settings` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric_settings
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Overview/KeyConcepts.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Overview/KeyConcepts.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+## _devlake_blueprint
+
+By adding the `project_name` field to the `blueprint` Models, we associate `project` with `blueprint` in one-to-one manner.
+# How to manage project via API
+
+For API specification, please check the swagger doc(by visiting `[Your Config-UI Host]/api/swagger/index.html`).
+Related endpoints:
+
+1. /projects
+2. /projects/:projectName/metrics
+3. /plugins
+
+# The interface that needs to be implemented
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+We divide plugins into three categories
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
+
+The second and third types of plug-ins here need to implement the corresponding interface to support the project.
+
+## Data Source Plugin
+
+`Data Source Plugin` is a plugin used to extract data from a data source.
+
+For example `GitLab` `GitHub` `Jira` `Jenkins` etc.
+
+These plugins, from various data sources, extract data into the database and store them, they deal directly with the data source, so we classify them as `Data Source Plugin`.
+
+## the DataSourcePluginBlueprintV200 interface
+
+`Data Source Plugin` needs to implement `DataSourcePluginBlueprintV200` interface to support `project`
+
+The interface definition for this interface is as follows
+
+```go
+// DataSourcePluginBlueprintV200 extends the V100 to provide support for
+// Project, so that complex metrics like DORA can be implemented based on a set
+// of Data Scopes
+type DataSourcePluginBlueprintV200 interface {
+ MakeDataSourcePipelinePlanV200(
+ connectionId uint64,
+ scopes []*BlueprintScopeV200,
+ syncPolicy BlueprintSyncPolicy,
+ ) (PipelinePlan, []Scope, errors.Error)
+}
+```
+
+`project` needs to provide a specific set of [Scopes](../Overview/KeyConcepts.md#data-scope) for a specific `connection` to the plug-in through this interface, and then obtain the plug-in involved in the `PipelineTask` All `plugins` and corresponding parameter information. At the same time, the plug-in needs to convert entities like `repo` and `board` in the data source into a `scope interface` that `project` can understand
+
+The corresponding `scope interface` has been implemented at following files of in the framework layer:
+- `models/domainlayer/ticket/board.go`
+- `models/domainlayer/code/repo.go`
+
+In the `plugins/gitlab/impl/impl.go` file, there is a `Gitlab` plugin implementation of the above interface, which can be used as a sample reference.
+
+And the `plugins/gitlab/api/blueprint_v200.go` contains implementation details.
+
+The following files contain the models that the relevant implementations depend on for reference:
+- `plugins/gitlab/models/project.go`
+- `plugins/gitlab/models/transformation_rule.go`
+
+## Metrics Plugin
+The `Metrics Plugin` is based on the data collected by the `Data Source Plugin` to do secondary calculations.
+
+For example `Dora`, and `Refdff` plugins belong to the `Metrics Plugin`
+
+These plugins are mainly for calculating various metrics, they do not directly contact the data source, so we classify them as `Metrics Plugin`.
+
+## The PluginMetric Interface
+
+`Metrics Plugin` needs to implement the `PluginMetric` interface to support `project`
+
+The interface definition for this interface looks like this:
+
+```go
+type PluginMetric interface {
+ // returns a list of required data entities and expected features.
+ // [{ "model": "cicd_tasks", "requiredFields": {"column": "type", "execptedValue": "Deployment"}}, ...]
+ RequiredDataEntities() (data []map[string]interface{}, err errors.Error)
+
+ // returns if the metric depends on Project for calculation.
+ // Currently, only dora would return true.
+ IsProjectMetric() bool
+
+ // indicates which plugins must be executed before executing this one.
+ // declare a set of dependencies with this
+ RunAfter() ([]string, errors.Error)
+
+ // returns an empty pointer of the plugin setting struct.
+ // (no concrete usage at this point)
+ Settings() (p interface{})
+}
+
+```
+
+`Project` needs `PluginMetric` to know whether a `Metrics Plugin` is dependent on `project`, and the tables and fields required in its calculation process.
Review Comment:
I think the code above is pretty self-explanatory, not need to say more.
However, a section link to "Refercences" is needed here:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] klesh commented on a diff in pull request #308: WIP Project framework
Posted by GitBox <gi...@apache.org>.
klesh commented on code in PR #308:
URL: https://github.com/apache/incubator-devlake-website/pull/308#discussion_r1011047351
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,195 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+---
+
+## Summary
+`Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
Review Comment:
Do not present `project` as some kind of subsystem for DORA, no.
Lets say it is **a set of scopes from different domains**, a way to group different resources, and it is crucial for some metric calculation like DORA.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,195 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+---
+
+## Summary
+`Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## project
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
Review Comment:
We can add a `project_mapping` to demonstrate how different scopes are connected
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,195 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+---
+
+## Summary
+`Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## project
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** |
+| ---------------- | --------------- | ---------------------- |
+| project_1 | gitlab | {'enable':true} |
+| project_2 | gitlab | {'enable':false} |
+| project_2 | github | {'enable':true} |
+
+
+It requires each plugin to implement an interface named `PluginMetric`
+
+# 如何使用 Project
+
+1. 首先我们可以通过 `GET` `/plugininfo` 接口来获取到 完整详细的 `插件` 信息，其中包括对应 `插件` 的每个表的结构信息。除此以外，我们也可以通过调用相对简单的 `GET` `/plugins` 来获取更为简洁的插件信息。简洁的插件信息仅包含插件名称，也就是`plugin_name`组成的列表，除此之外不包含其他任何信息。
Review Comment:
The order of the steps doesn't feel right, my first reaction to this API was: why am I looking at `/plugininfo` ? I'm reading the project document...
Let's start from creating project, and then how to enable metrics for the project, and then `/plugins` for fetching all metric plugin names
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,195 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+---
+
+## Summary
+`Project` is an object that can be used for DORA to associate `changes(commits)`, with `deployments` and `issues`.
+
+It contains the following two models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes the mapping relationship enabled by a plugin, including the name of the project and plugin on both sides of the mapping, and basic information about plugin options.
+
+
+
+## project
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metric
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** |
+| ---------------- | --------------- | ---------------------- |
+| project_1 | gitlab | {'enable':true} |
Review Comment:
Oh, maybe we should extract the `enable` as a `field`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] mappjzc commented on a diff in pull request #308: WIP Project framework
Posted by GitBox <gi...@apache.org>.
mappjzc commented on code in PR #308:
URL: https://github.com/apache/incubator-devlake-website/pull/308#discussion_r1034716626
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like `Dora`, we need to correctly associate different [Scope](../Glossary.md#data-scope) under different data sources through a method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate `project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
+
+By visiting `[Your Config-UI Host]/api/swagger/index.html`, you can go to the swagger document of `Devlake` deployed. In this document, `framework/projects` `framework/ProjectMetrics`, and `framework/plugins ` is the related API of `project`.
+
+To protect the integrity of the data in the database as much as possible, we recommend using the API to perform corresponding operations instead of directly using SQL statements to operate the tables in the database.
+
+When we need to manipulate the `projects` table, we can use the `framework/projects`:
+- We can create a new `project` object through the `POST` `/projects` interface, which requires a `projectName` field as its name, which is unique.
+
+After we created the `project` object or we already had a `project`:
+- We can use the `GET /projects/:projectName` interface to obtain specific `project` information. In addition to the name, this information also includes the update time and creation time of the project, and will be associated with all The relevant `project_metrics` and `project_mapping` information of the current `project` will be returned after integration.
+- We can use the `PATCH /projects/:projectName` interface to update the information of a specific `project`, such as updating its related `description` information.
+- We can get all `project` information through the `GET /projects` interface. The returned information can be filtered by setting the parameter `search`, and the returned information can be paged by setting the parameters `page`, and `pageSize`.
+
+After we created the `project` object or we already had a `project`:
+- we can use the `POST /project_metrics/:projectName/:pluginName` to create a set of relationships between `project` and `Metrics Plugin`, Use the `projectName` and `pluginName` we obtained earlier to build this relationship. At the same time configure the corresponding `option` for it.
+
+After we create a `project_metrics` relation or we already had one:
+- We can use the `GET /project_metrics/:projectName/:pluginName` to get a set of `project_metrics` information.
+- We can use the `PATCH /project_metrics/:projectName/:pluginName` to update the `project_metrics` information.
+
+If we need to know which plugins are `Metrics Plugin`:
+- we can use the `GET` `/plugins`, it will return a list of all plugin names and information about the plugins and `Metrics Plugin`.
+
+# The interface that needs to be implemented
+
+We divide plugins into three categories
+- The first category is `Base Plugin`, such as `core`, `helper`, etc. These plug-ins provide basic support services for other plug-ins.
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
+
+The second and third types of plug-ins here need to implement the corresponding interface to support the project.
+
+## Data Source Plugin
+
+`Data Source Plugin` is a plugin used to extract data from a data source.
+
+For example `GitLab` `GitHub` `Jira` `Jenkins` etc.
+
+These plugins, from various data sources, extract data into the database and store them, they deal directly with the data source, so we classify them as `Data Source Plugin`.
+
+## the DataSourcePluginBlueprintV200 interface
Review Comment:
this is arealdy on "Interface" section
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] mappjzc commented on a diff in pull request #308: WIP Project framework
Posted by GitBox <gi...@apache.org>.
mappjzc commented on code in PR #308:
URL: https://github.com/apache/incubator-devlake-website/pull/308#discussion_r1034715885
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter situations requiring comprehensive calculations based on data from multiple data sources.
+
+For example, we do code version management on `Gitlab`, and then do corresponding compilation and deployment operations in `Jenkins`. At this time, our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like `Dora`, we need to correctly associate different [Scope](../Glossary.md#data-scope) under different data sources through a method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different domains**, a way to group different resources, and it is crucial for some metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate `project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some other operations that need to be associated with `Project` to add, delete, and modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope, including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** | **key** |
+| ------------- | -------- | ---------- | ----------------------------- | ------- |
+| `name` | varchar | 255 | name for project | PK |
+| `description` | longtext | | description of the project | |
+| `created_at` | datetime | 3 | created time of project | |
+| `updated_at` | datetime | 3 | last updated time of project | |
+
+
+| **name** | **describe** | **created_at** | **updated_at** |
+| --------- | ------------------------------------ | ----------------------- | ------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 | 2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 | 2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description** | **key** |
+| --------------- | -------- | ---------- | ---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `plugin_name` | varchar | 255 | name for plugin | PK |
+| `plugin_option` | longtext | | check if metric plugins have been enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled | |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description** | **key** |
+| -------------- | -------- | ---------- | ------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project | PK |
+| `table` | varchar | 255 | the table name of [Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the [Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
+
+By visiting `[Your Config-UI Host]/api/swagger/index.html`, you can go to the swagger document of `Devlake` deployed. In this document, `framework/projects` `framework/ProjectMetrics`, and `framework/plugins ` is the related API of `project`.
+
+To protect the integrity of the data in the database as much as possible, we recommend using the API to perform corresponding operations instead of directly using SQL statements to operate the tables in the database.
+
+When we need to manipulate the `projects` table, we can use the `framework/projects`:
+- We can create a new `project` object through the `POST` `/projects` interface, which requires a `projectName` field as its name, which is unique.
+
+After we created the `project` object or we already had a `project`:
+- We can use the `GET /projects/:projectName` interface to obtain specific `project` information. In addition to the name, this information also includes the update time and creation time of the project, and will be associated with all The relevant `project_metrics` and `project_mapping` information of the current `project` will be returned after integration.
+- We can use the `PATCH /projects/:projectName` interface to update the information of a specific `project`, such as updating its related `description` information.
+- We can get all `project` information through the `GET /projects` interface. The returned information can be filtered by setting the parameter `search`, and the returned information can be paged by setting the parameters `page`, and `pageSize`.
+
+After we created the `project` object or we already had a `project`:
+- we can use the `POST /project_metrics/:projectName/:pluginName` to create a set of relationships between `project` and `Metrics Plugin`, Use the `projectName` and `pluginName` we obtained earlier to build this relationship. At the same time configure the corresponding `option` for it.
+
+After we create a `project_metrics` relation or we already had one:
+- We can use the `GET /project_metrics/:projectName/:pluginName` to get a set of `project_metrics` information.
+- We can use the `PATCH /project_metrics/:projectName/:pluginName` to update the `project_metrics` information.
+
+If we need to know which plugins are `Metrics Plugin`:
+- we can use the `GET` `/plugins`, it will return a list of all plugin names and information about the plugins and `Metrics Plugin`.
+
+# The interface that needs to be implemented
+
+We divide plugins into three categories
+- The first category is `Base Plugin`, such as `core`, `helper`, etc. These plug-ins provide basic support services for other plug-ins.
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub` `Jira` `Jenkins`, etc. These plugins collect data from various data sources
Review Comment:
this is arealdy on "Interface" section
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org