You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by zk...@apache.org on 2023/05/19 14:35:58 UTC
[incubator-devlake-website] branch main updated: docs: update annotations and wordings for DORA metrics in v0.17 and main (#538)
This is an automated email from the ASF dual-hosted git repository.
zky pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-devlake-website.git
The following commit(s) were added to refs/heads/main by this push:
new 32325bd348 docs: update annotations and wordings for DORA metrics in v0.17 and main (#538)
32325bd348 is described below
commit 32325bd3484145167c8fd8218cd4b37181a3f211
Author: Louis.z <lo...@gmail.com>
AuthorDate: Fri May 19 22:35:54 2023 +0800
docs: update annotations and wordings for DORA metrics in v0.17 and main (#538)
* fix: add visual description for metrics
* fix: update the desc for required data and tr
---------
Co-authored-by: Startrekzky <ka...@merico.dev>
---
.../Configuration/images/deployment_definition.png | Bin 0 -> 150056 bytes
.../images/lead_time_for_changes_definition.png | Bin 0 -> 397572 bytes
docs/Metrics/CFR.md | 16 +++-------------
docs/Metrics/DeploymentFrequency.md | 14 ++++++--------
docs/Metrics/LeadTimeForChanges.md | 21 +++++++++------------
docs/Metrics/MTTR.md | 16 +++-------------
.../Configuration/images/deployment_definition.png | Bin 0 -> 150056 bytes
.../images/lead_time_for_changes_definition.png | Bin 0 -> 397572 bytes
versioned_docs/version-v0.17/Metrics/CFR.md | 14 +++-----------
.../version-v0.17/Metrics/DeploymentFrequency.md | 16 ++++++++--------
.../version-v0.17/Metrics/LeadTimeForChanges.md | 19 +++++++++----------
versioned_docs/version-v0.17/Metrics/MTTR.md | 14 +++-----------
12 files changed, 44 insertions(+), 86 deletions(-)
diff --git a/docs/Configuration/images/deployment_definition.png b/docs/Configuration/images/deployment_definition.png
new file mode 100644
index 0000000000..7c54fd4c5f
Binary files /dev/null and b/docs/Configuration/images/deployment_definition.png differ
diff --git a/docs/Configuration/images/lead_time_for_changes_definition.png b/docs/Configuration/images/lead_time_for_changes_definition.png
new file mode 100644
index 0000000000..fa10c990f2
Binary files /dev/null and b/docs/Configuration/images/lead_time_for_changes_definition.png differ
diff --git a/docs/Metrics/CFR.md b/docs/Metrics/CFR.md
index 019b9b1e8a..7aa2dd2b7e 100644
--- a/docs/Metrics/CFR.md
+++ b/docs/Metrics/CFR.md
@@ -38,22 +38,12 @@ Below are the benchmarks for different development teams from Google's report. H
<b>Data Sources Required</b>
-This metric relies on:
-
-- `Deployments` collected in one of the following ways:
- - Open APIs of Jenkins, GitLab, GitHub, etc.
- - Webhook for general CI tools.
- - Releases and PR/MRs from GitHub, GitLab APIs, etc.
-- `Incidents` collected in one of the following ways:
- - Issue tracking tools such as Jira, TAPD, GitHub, etc.
- - Incident or Service Monitoring tools such as PagerDuty, ServiceNow, etc.
+- `Deployments` from Jenkins, GitLab CI, GitHub Action, BitBucket Pipelines, or Webhook, etc.
+- `Incidents` from Jira issues, GitHub issues, TAPD issues, PagerDuty Incidents, etc.
<b>Transformation Rules Required</b>
-This metric relies on:
-
-- Deployment configuration in Jenkins, GitLab, GitHub or BitBucket transformation rules to let DevLake know which CI builds/jobs can be regarded as `Deployments`.
-- Incident configuration in Jira, GitHub or TAPD transformation rules to let DevLake know which issues can be regarded as `Incidents`.
+Define `deployment` and `incident` in [data transformations](https://devlake.apache.org/docs/Configuration/Tutorial#step-3---add-transformations-optional) while configuring the blueprint of a project to let DevLake know what CI/issue records can be regarded as deployments or incidents.
<b>SQL Queries</b>
diff --git a/docs/Metrics/DeploymentFrequency.md b/docs/Metrics/DeploymentFrequency.md
index 4d69401527..0e105f98fb 100644
--- a/docs/Metrics/DeploymentFrequency.md
+++ b/docs/Metrics/DeploymentFrequency.md
@@ -7,7 +7,9 @@ sidebar_position: 26
## What is this metric?
-How often an organization deploys code to production or release it to end users.
+How often an organization deploys code to production or release it to end users. Below is a picture showing the definition of DevLake `deployments`.
+
+
## Why is it important?
@@ -36,19 +38,15 @@ Below are the benchmarks for different development teams from Google's report. D
<b>Data Sources Required</b>
-This metric relies on deployments collected in multiple ways:
-
-- Open APIs of Jenkins, GitLab, GitHub, etc.
-- Webhook for general CI tools.
-- Releases and PR/MRs from GitHub, GitLab APIs, etc.
+`Deployments` from Jenkins, GitLab CI, GitHub Action, BitBucket Pipelines, Webhook, etc.
<b>Transformation Rules Required</b>
-This metric relies on the deployment configuration in Jenkins, GitLab or GitHub transformation rules to let DevLake know what CI builds/jobs can be regarded as deployments.
+Define `deployment` in [data transformations](https://devlake.apache.org/docs/Configuration/Tutorial#step-3---add-transformations-optional) while configuring the blueprint of a project to let DevLake know what CI records can be regarded as deployments.
<b>SQL Queries</b>
-If you want to measure the monthly trend of deployment count as the picture shown below, run the following SQL in Grafana.
+DevLake deployments can be found in table [cicd_deployment_commits](/docs/DataModels/DevLakeDomainLayerSchema.md#cicd_deployment_commits). If you want to measure the monthly trend of deployment count as the picture shown below, run the following SQL in Grafana.
diff --git a/docs/Metrics/LeadTimeForChanges.md b/docs/Metrics/LeadTimeForChanges.md
index 33ae8a3a37..f18ec3963b 100644
--- a/docs/Metrics/LeadTimeForChanges.md
+++ b/docs/Metrics/LeadTimeForChanges.md
@@ -1,7 +1,7 @@
---
-title: "DORA - Lead Time for Changes"
+title: "DORA - Median Lead Time for Changes"
description: >
- DORA - Lead Time for Changes
+ DORA - Median Lead Time for Changes
sidebar_position: 27
---
@@ -18,13 +18,13 @@ This metric measures the time it takes to a code change to the production enviro
DORA dashboard. See [live demo](https://grafana-lake.demo.devlake.io/grafana/d/qNo8_0M4z/dora?orgId=1).
## How is it calculated?
-
This metric is quite similar to [PR Cycle Time](PRCycleTime.md). The difference is that 'Lead Time for Changes' uses a different method to filter PRs.
-1. Find the PRs' associated deployments whose finished_date falls into the time range that users select
-2. Calculate the PRs' median cycle time. This will be the Median Lead Time for Changes.
+1. Find the PRs' associated deployment commits whose finished_date falls into the time range that users select.
+2. Find the associated pull requests of the commits diff between two consecutive successful `deployment commits` in the production environment.
+3. Calculate the PRs' median cycle time. This will be the Median Lead Time for Changes.
-
+
PR cycle time is pre-calculated by the `dora` plugin during every data collection. You can find it in `pr_cycle_time` in [table.project_pr_metrics](https://devlake.apache.org/docs/DataModels/DevLakeDomainLayerSchema/#project_pr_metrics) of DevLake's database.
@@ -41,15 +41,12 @@ Below are the benchmarks for different development teams from Google's report. H
<b>Data Sources Required</b>
-This metric relies on deployments collected in multiple ways:
-
-- Open APIs of Jenkins, GitLab, GitHub, etc.
-- Webhook for general CI tools.
-- Releases and PR/MRs from GitHub, GitLab APIs, etc.
+- `Deployments` from Jenkins, GitLab CI, GitHub Action, BitBucket Pipelines, Webhook, etc.
+- `Pull Requests` from GitHub PRs, GitLab MRs, BitBucket PRs, Azure DevOps PRs, etc.
<b>Transformation Rules Required</b>
-This metric relies on the deployment configuration in Jenkins, GitLab or GitHub transformation rules to let DevLake know what CI builds/jobs can be regarded as deployments.
+Define `deployment` in [data transformations](https://devlake.apache.org/docs/Configuration/Tutorial#step-3---add-transformations-optional) while configuring the blueprint of a project to let DevLake know what CI records can be regarded as deployments.
<b>SQL Queries</b>
diff --git a/docs/Metrics/MTTR.md b/docs/Metrics/MTTR.md
index 75fe938e64..34829efdc2 100644
--- a/docs/Metrics/MTTR.md
+++ b/docs/Metrics/MTTR.md
@@ -36,22 +36,12 @@ Below are the benchmarks for different development teams from Google's report. H
<b>Data Sources Required</b>
-This metric relies on:
-
-- `Deployments` collected in one of the following ways:
- - Open APIs of Jenkins, GitLab, GitHub, etc.
- - Webhook for general CI tools.
- - Releases and PR/MRs from GitHub, GitLab APIs, etc.
-- `Incidents` collected in one of the following ways:
- - Issue tracking tools such as Jira, TAPD, GitHub, etc.
- - Incident or Service Monitoring tools such as PagerDuty, ServiceNow, etc.
+- `Deployments` from Jenkins, GitLab CI, GitHub Action, BitBucket Pipelines, or Webhook, etc.
+- `Incidents` from Jira issues, GitHub issues, TAPD issues, PagerDuty Incidents, etc.
<b>Transformation Rules Required</b>
-This metric relies on:
-
-- Deployment configuration in Jenkins, GitLab or GitHub transformation rules to let DevLake know what CI builds/jobs can be regarded as `Deployments`.
-- Incident configuration in Jira, GitHub or TAPD transformation rules to let DevLake know what CI builds/jobs can be regarded as `Incidents`.
+Define `deployment` and `incident` in [data transformations](https://devlake.apache.org/docs/Configuration/Tutorial#step-3---add-transformations-optional) while configuring the blueprint of a project to let DevLake know what CI/issue records can be regarded as deployments or incidents.
<b>SQL Queries</b>
diff --git a/versioned_docs/version-v0.17/Configuration/images/deployment_definition.png b/versioned_docs/version-v0.17/Configuration/images/deployment_definition.png
new file mode 100644
index 0000000000..7c54fd4c5f
Binary files /dev/null and b/versioned_docs/version-v0.17/Configuration/images/deployment_definition.png differ
diff --git a/versioned_docs/version-v0.17/Configuration/images/lead_time_for_changes_definition.png b/versioned_docs/version-v0.17/Configuration/images/lead_time_for_changes_definition.png
new file mode 100644
index 0000000000..fa10c990f2
Binary files /dev/null and b/versioned_docs/version-v0.17/Configuration/images/lead_time_for_changes_definition.png differ
diff --git a/versioned_docs/version-v0.17/Metrics/CFR.md b/versioned_docs/version-v0.17/Metrics/CFR.md
index 64cc22e010..0f3fe218ad 100644
--- a/versioned_docs/version-v0.17/Metrics/CFR.md
+++ b/versioned_docs/version-v0.17/Metrics/CFR.md
@@ -35,20 +35,12 @@ Below are the benchmarks for different development teams from Google's report. H
<b>Data Sources Required</b>
-This metric relies on:
-- `Deployments` collected in one of the following ways:
- - Open APIs of Jenkins, GitLab, GitHub, etc.
- - Webhook for general CI tools.
- - Releases and PR/MRs from GitHub, GitLab APIs, etc.
-- `Incidents` collected in one of the following ways:
- - Issue tracking tools such as Jira, TAPD, GitHub, etc.
- - Incident or Service Monitoring tools such as PagerDuty, ServiceNow, etc.
+- `Deployments` from Jenkins, GitLab CI, GitHub Action, BitBucket Pipelines, or Webhook, etc.
+- `Incidents` from Jira issues, GitHub issues, TAPD issues, PagerDuty Incidents, etc.
<b>Transformation Rules Required</b>
-This metric relies on:
-- Deployment configuration in Jenkins, GitLab, GitHub or BitBucket transformation rules to let DevLake know which CI builds/jobs can be regarded as `Deployments`.
-- Incident configuration in Jira, GitHub or TAPD transformation rules to let DevLake know which issues can be regarded as `Incidents`.
+Define `deployment` and `incident` in [data transformations](https://devlake.apache.org/docs/Configuration/Tutorial#step-3---add-transformations-optional) while configuring the blueprint of a project to let DevLake know what CI/issue records can be regarded as deployments or incidents.
<b>SQL Queries</b>
diff --git a/versioned_docs/version-v0.17/Metrics/DeploymentFrequency.md b/versioned_docs/version-v0.17/Metrics/DeploymentFrequency.md
index 2aa57db4d2..e35e3e2750 100644
--- a/versioned_docs/version-v0.17/Metrics/DeploymentFrequency.md
+++ b/versioned_docs/version-v0.17/Metrics/DeploymentFrequency.md
@@ -5,8 +5,11 @@ description: >
sidebar_position: 26
---
-## What is this metric?
-How often an organization deploys code to production or release it to end users.
+## What is this metric?
+
+How often an organization deploys code to production or release it to end users. Below is a picture showing the definition of DevLake `deployments`.
+
+
## Why is it important?
Deployment frequency reflects the efficiency of a team's deployment. A team that deploys more frequently can deliver the product faster and users' feature requirements can be met faster.
@@ -34,18 +37,15 @@ Below are the benchmarks for different development teams from Google's report. D
<b>Data Sources Required</b>
-This metric relies on deployments collected in multiple ways:
-- Open APIs of Jenkins, GitLab, GitHub, etc.
-- Webhook for general CI tools.
-- Releases and PR/MRs from GitHub, GitLab APIs, etc.
+`Deployments` from Jenkins, GitLab CI, GitHub Action, BitBucket Pipelines, Webhook, etc.
<b>Transformation Rules Required</b>
-This metric relies on the deployment configuration in Jenkins, GitLab or GitHub transformation rules to let DevLake know what CI builds/jobs can be regarded as deployments.
+Define `deployment` in [data transformations](https://devlake.apache.org/docs/Configuration/Tutorial#step-3---add-transformations-optional) while configuring the blueprint of a project to let DevLake know what CI records can be regarded as deployments.
<b>SQL Queries</b>
-If you want to measure the monthly trend of deployment count as the picture shown below, run the following SQL in Grafana.
+DevLake deployments can be found in table [cicd_deployment_commits](/docs/DataModels/DevLakeDomainLayerSchema.md#cicd_deployment_commits). If you want to measure the monthly trend of deployment count as the picture shown below, run the following SQL in Grafana.
diff --git a/versioned_docs/version-v0.17/Metrics/LeadTimeForChanges.md b/versioned_docs/version-v0.17/Metrics/LeadTimeForChanges.md
index d96d194861..0fe8e26fc9 100644
--- a/versioned_docs/version-v0.17/Metrics/LeadTimeForChanges.md
+++ b/versioned_docs/version-v0.17/Metrics/LeadTimeForChanges.md
@@ -1,7 +1,7 @@
---
-title: "DORA - Lead Time for Changes"
+title: "DORA - Median Lead Time for Changes"
description: >
- DORA - Lead Time for Changes
+ DORA - Median Lead Time for Changes
sidebar_position: 27
---
@@ -18,10 +18,11 @@ DORA dashboard. See [live demo](https://grafana-lake.demo.devlake.io/grafana/d/q
## How is it calculated?
This metric is quite similar to [PR Cycle Time](PRCycleTime.md). The difference is that 'Lead Time for Changes' uses a different method to filter PRs.
-1. Find the PRs' associated deployments whose finished_date falls into the time range that users select
-2. Calculate the PRs' median cycle time. This will be the Median Lead Time for Changes.
+1. Find the PRs' associated deployment commits whose finished_date falls into the time range that users select.
+2. Find the associated pull requests of the commits diff between two consecutive successful `deployment commits` in the production environment.
+3. Calculate the PRs' median cycle time. This will be the Median Lead Time for Changes.
-
+
PR cycle time is pre-calculated by the `dora` plugin during every data collection. You can find it in `pr_cycle_time` in [table.project_pr_metrics](https://devlake.apache.org/docs/DataModels/DevLakeDomainLayerSchema/#project_pr_metrics) of DevLake's database.
@@ -39,14 +40,12 @@ Below are the benchmarks for different development teams from Google's report. H
<b>Data Sources Required</b>
-This metric relies on deployments collected in multiple ways:
-- Open APIs of Jenkins, GitLab, GitHub, etc.
-- Webhook for general CI tools.
-- Releases and PR/MRs from GitHub, GitLab APIs, etc.
+- `Deployments` from Jenkins, GitLab CI, GitHub Action, BitBucket Pipelines, Webhook, etc.
+- `Pull Requests` from GitHub PRs, GitLab MRs, BitBucket PRs, Azure DevOps PRs, etc.
<b>Transformation Rules Required</b>
-This metric relies on the deployment configuration in Jenkins, GitLab or GitHub transformation rules to let DevLake know what CI builds/jobs can be regarded as deployments.
+Define `deployment` in [data transformations](https://devlake.apache.org/docs/Configuration/Tutorial#step-3---add-transformations-optional) while configuring the blueprint of a project to let DevLake know what CI records can be regarded as deployments.
<b>SQL Queries</b>
diff --git a/versioned_docs/version-v0.17/Metrics/MTTR.md b/versioned_docs/version-v0.17/Metrics/MTTR.md
index 7465a99e2b..a5d3117157 100644
--- a/versioned_docs/version-v0.17/Metrics/MTTR.md
+++ b/versioned_docs/version-v0.17/Metrics/MTTR.md
@@ -33,20 +33,12 @@ Below are the benchmarks for different development teams from Google's report. H
<b>Data Sources Required</b>
-This metric relies on:
-- `Deployments` collected in one of the following ways:
- - Open APIs of Jenkins, GitLab, GitHub, etc.
- - Webhook for general CI tools.
- - Releases and PR/MRs from GitHub, GitLab APIs, etc.
-- `Incidents` collected in one of the following ways:
- - Issue tracking tools such as Jira, TAPD, GitHub, etc.
- - Incident or Service Monitoring tools such as PagerDuty, ServiceNow, etc.
+- `Deployments` from Jenkins, GitLab CI, GitHub Action, BitBucket Pipelines, or Webhook, etc.
+- `Incidents` from Jira issues, GitHub issues, TAPD issues, PagerDuty Incidents, etc.
<b>Transformation Rules Required</b>
-This metric relies on:
-- Deployment configuration in Jenkins, GitLab or GitHub transformation rules to let DevLake know what CI builds/jobs can be regarded as `Deployments`.
-- Incident configuration in Jira, GitHub or TAPD transformation rules to let DevLake know what CI builds/jobs can be regarded as `Incidents`.
+Define `deployment` and `incident` in [data transformations](https://devlake.apache.org/docs/Configuration/Tutorial#step-3---add-transformations-optional) while configuring the blueprint of a project to let DevLake know what CI/issue records can be regarded as deployments or incidents.
<b>SQL Queries</b>