You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by GitBox <gi...@apache.org> on 2022/11/16 07:47:50 UTC
[GitHub] [incubator-devlake] klesh commented on a diff in pull request #3742: feat(dora): update logic for change lead time
klesh commented on code in PR #3742:
URL: https://github.com/apache/incubator-devlake/pull/3742#discussion_r1023597405
##########
plugins/dora/e2e/raw_tables/cicd_tasks_changeleadtime.csv:
##########
@@ -0,0 +1,19 @@
+id,created_at,updated_at,_raw_data_params,_raw_data_table,_raw_data_id,_raw_data_remark,name,pipeline_id,result,status,type,environment,duration_sec,started_date,finished_date,cicd_scope_id
Review Comment:
Does the lengthy `_raw_data_remark` serve any purpose in the test case? It is hard to review, can we shorten the content of it?
I see
When I view it in LibreOffice, the 6th row didn't display correctly.
The biggest problem is that all `cicd_tasks` are `DEPLOYMENT`, please add other kinds of `cicd_task` between deployment to mimic actually data.
##########
plugins/dora/e2e/raw_tables/commits_diffs.csv:
##########
@@ -0,0 +1,21 @@
+commit_sha,new_commit_sha,old_commit_sha,sorting_index
Review Comment:
Please sort them by `new_commit_sha` / `old_commit_sha` and then `commit_sha` for the sake of the reviewers.
##########
plugins/dora/e2e/raw_tables/project_mapping.csv:
##########
@@ -0,0 +1,4 @@
+project_name,table,row_id,created_at,updated_at,_raw_data_params,_raw_data_table,_raw_data_id,_raw_data_remark
+project1,cicd_scopes,repo2,2019-09-16 14:32:15.000,2015-03-30 03:40:02.000,OKUc0VfGz4,VuOk5CsjZH,155,"Navicat Monitor requires a repository to store alerts and metrics for historical analysis. I may not have gone where I intended to go, but I think I have ended up where I needed to be. To successfully establish a new connection to local/remote server - no matter via SSL, SSH or HTTP, set the database login information in the General tab. What you get by achieving your goals is not as important as what you become by achieving your goals. If you wait, all that happens is you get older. Typically, it is employed as an encrypted version of Telnet. In other words, Navicat provides the ability for data in different databases and/or schemas to be kept up-to-date so that each repository contains the same information. After logged in the Navicat Cloud feature, the Navigation pane will be divided into Navicat Cloud and My Connections sections. Navicat Data Modeler is a powerful and cost-effec
tive database design tool which helps you build high-quality conceptual, logical and physical data models. A man’s best friends are his ten fingers. In a Telnet session, all communications, including username and password, are transmitted in plain-text, allowing anyone to listen-in on your session and steal passwords and other information. You can select any connections, objects or projects, and then select the corresponding buttons on the Information Pane. To get a secure connection, the first thing you need to do is to install OpenSSL Library and download Database Source. There is no way to happiness. Happiness is the way. If you wait, all that happens is you get older. How we spend our days is, of course, how we spend our lives. Navicat provides powerful tools for working with queries: Query Editor for editing the query text directly, and Query Builder, Find Builder or Aggregate Builder for building queries visually. Optimism is the one quality more associated with success and
happiness than any other. There is no way to happiness. Happiness is the way. Monitored servers include MySQL, MariaDB and SQL Server, and compatible with cloud databases like Amazon RDS, Amazon Aurora, Oracle Cloud, Google Cloud and Microsoft Azure. You cannot save people, you can just love them. To get a secure connection, the first thing you need to do is to install OpenSSL Library and download Database Source. Navicat Cloud could not connect and access your databases. By which it means, it could only store your connection settings, queries, model files, and virtual group; your database passwords and data (e.g. tables, views, etc) will not be stored to Navicat Cloud. Creativity is intelligence having fun. What you get by achieving your goals is not as important as what you become by achieving your goals. After comparing data, the window shows the number of records that will be inserted, updated or deleted in the target. Sometimes you win, sometimes you learn. Navicat Cloud could
not connect and access your databases. By which it means, it could only store your connection settings, queries, model files, and virtual group; your database passwords and data (e.g. tables, views, etc) will not be stored to Navicat Cloud. Creativity is intelligence having fun."
Review Comment:
For Change Lead Time calculation, I presume that we first need to filter out all PR by the project first.
How is that possible when the `project1` doesn't contain any record for the `repo`?
##########
plugins/dora/e2e/raw_tables/project_mapping.csv:
##########
@@ -0,0 +1,4 @@
+project_name,table,row_id,created_at,updated_at,_raw_data_params,_raw_data_table,_raw_data_id,_raw_data_remark
Review Comment:
`_raw_xxx` fields are irrelevant in the DORA metric, I think we can omit these fields, same as above.
##########
plugins/dora/e2e/raw_tables/pull_requests.csv:
##########
@@ -1,10 +1,10 @@
-id,created_at,updated_at,_raw_data_params,_raw_data_table,_raw_data_id,_raw_data_remark,base_repo_id,head_repo_id,status,title,description,url,author_name,author_id,parent_pr_id,pull_request_key,created_date,merged_date,closed_date,type,component,merge_commit_sha,head_ref,base_ref,base_commit_sha,head_commit_sha,coding_timespan,review_lag,review_timespan,deploy_timespan,change_timespan,orig_coding_timespan,orig_review_lag,orig_review_timespan,orig_deploy_timespan
-github:GithubPullRequest:1:1043463302,2022-09-15 05:36:39.856,2022-09-15 05:36:39.856,"{""ConnectionId"":1,""Owner"":""apache"",""Repo"":""incubator-devlake""}",_raw_github_api_pull_requests,13176,,github:GithubRepo:1:384111310,github:GithubRepo:1:384111310,closed,feat: implement API for plugin `customize`,"""# Summary\r\n\r\nfulfill the requirement \r\n#2880 ([Feature][customize] implement API for plugin customize) \r\n#2985 ([Feature][customize] new sub-task ExtractCustomizedFields for plugin customize)\r\n\r\nrelated to #2802 \r\nWe take Jira issue as an example.\r\n\r\n1. find all primary keys of table `issues`, which is `id`\r\n2. retrieve data from `issues` and ` _raw_jira_api_issues`\r\n```SQL\r\nSELECT issues._raw_data_id, _raw_jira_api_issues.data, issues.id FROM `issues` LEFT JOIN _raw_jira_api_issues ON issues._raw_data_id = _raw_jira_api_issues.id WHERE _raw_data_table = '_raw_jira_api_issues' AND _raw_data_params = '{\""ConnectionId\"":1,\""BoardId\"":8}'\r\n```\r\n3.
extract values from `_raw_jira_api_issues.data` \r\n4. assign them to the columns in `issues`\r\n\r\n### Does this close any open issues?\r\nCloses #2880 \r\nCloses #2985 \r\n\r\n### Screenshots\r\nInclude any relevant screenshots here.\r\n\r\n### Other Information\r\nAn example of `TaskOptions`\r\n```json\r\n{\r\n \""transformationRules\"":[ \r\n {\r\n \""table\"":\""issues\"", // domain layer table name\r\n \""rawDataTable\"":\""_raw_jira_api_issues\"", // raw layer table, from which we extract values by json path\r\n \""rawDataParams\"":\""{\\\""ConnectionId\\\"":1,\\\""BoardId\\\"":8}\"", // the value should be a string\r\n \""mapping\"":{\r\n \""x_test\"":\""fields.created\"" // the key is extension field name, the value is json path\r\n }\r\n }\r\n ]\r\n}\r\n```""",https://github.com/apache/incubator-devlake/pull/2911,mindlesscloud,github:GithubAccount:1:8455907,,2911,2022-09-01 09:34:40,2022-09-09 07:52:53,20
22-09-09 07:52:54,,,075e3d3b480e407aede5a0893bd61de9b0fd051d,customize-plugin,main,0006e8105d70318aff5eeee38d405fa181a32aa0,a57007bfeda7b6a15f2ce7901422ca6c8cdc8ede,,,,,,0,0,0,0
-github:GithubPullRequest:1:1048233599,2022-09-15 05:36:39.856,2022-09-15 05:36:39.856,"{""ConnectionId"":1,""Owner"":""apache"",""Repo"":""incubator-devlake""}",_raw_github_api_pull_requests,13211,,github:GithubRepo:1:384111310,github:GithubRepo:1:384111310,closed,feat: new plugin for gitea,"""# Summary\r\n[Gitea](https://gitea.io/) is an open-source software package for hosting software development version control using Git as well as other collaborative features like bug tracking, code review, kanban boards, tickets, and wikis.\r\n<!--\r\nThanks for submitting a pull request!\r\n\r\nWe appreciate you spending the time to work on these changes.\r\nPlease fill out as many sections below as possible.\r\n-->\r\n\r\n### Does this close any open issues?\r\nCloses [2992](https://github.com/apache/incubator-devlake/issues/2992)\r\n\r\n### Screenshots\r\nInclude any relevant screenshots here.\r\n\r\n### Other Information\r\nAny other information that is important to this PR.\r\n""",https:/
/github.com/apache/incubator-devlake/pull/2991,tk103331,github:GithubAccount:1:4404609,,2991,2022-09-07 04:07:39,2022-09-10 02:35:43,2022-09-10 02:35:44,,,3b6c19449a8e7646a31ee2d170df5c55252e5879,gitea-plugin,feat-plugin-gitea,ef013b890f53b98b41611f508554c7084d6b3a28,d715995e019be9ad12c5fed99140303afa967e28,,,,,,0,0,0,0
-github:GithubPullRequest:1:1049191985,2022-09-15 05:36:40.170,2022-09-15 05:36:40.170,"{""ConnectionId"":1,""Owner"":""apache"",""Repo"":""incubator-devlake""}",_raw_github_api_pull_requests,12680,,github:GithubRepo:1:384111310,github:GithubRepo:1:384111310,closed,[issue-2908]: Bump lake-builder version to 0.0.8,"""### ⚠️ Pre Checklist\r\n\r\n> Please complete _ALL_ items in this checklist, and remove before submitting\r\n\r\n- [x] I have read through the [Contributing Documentation](https://devlake.apache.org/community/).\r\n- [x] I have added relevant tests.\r\n- [x] I have added relevant documentation.\r\n- [ ] I will add labels to the PR, such as `pr-type/bug-fix`, `pr-type/feature-development`, etc.\r\n\r\n\r\n\r\n# Summary\r\n\r\n<!--\r\nThanks for submitting a pull request!\r\n\r\nWe appreciate you spending the time to work on these changes.\r\nPlease fill out as many sections below as possible.\r\n-->\r\n\r\n### Does this close any open issues?\r\nCloses #2908 \r\nBlocks
#2940/#2984\r\nBlocks #2907/#2941\r\n\r\n### Screenshots\r\nInclude any relevant screenshots here.\r\n\r\n### Other Information\r\nAny other information that is important to this PR.\r\n""",https://github.com/apache/incubator-devlake/pull/3012,keon94,github:GithubAccount:1:25063936,,3012,2022-09-07 20:30:44,2022-09-09 03:39:50,2022-09-09 03:39:50,,,285c0afd732639d49cc5d0eb7de75c6fa0623cc0,builder-v0.0.8,main,7d0dfb888e2fb5b79cd4654186214e9210e72e98,0edd5d0e0d18516d3309fec90553c3ee7380c149,,,,,,0,0,0,0
-github:GithubPullRequest:1:1051243958,2022-09-15 05:36:40.170,2022-09-15 05:36:40.170,"{""ConnectionId"":1,""Owner"":""apache"",""Repo"":""incubator-devlake""}",_raw_github_api_pull_requests,12691,,github:GithubRepo:1:384111310,github:GithubRepo:1:384111310,closed,fix: golangci-lint error,"""# Summary\r\n\r\ngolint-ci was down for a while, this PR try to fix all missed linting errors during the period.\r\n\r\n### Screenshots\r\n\r\n\r\n\r\n""",https://github.com/apache/incubator-devlake/pull/3032,klesh,github:GithubAccount:1:61080,,3032,2022-09-09 08:01:07,2022-09-09 17:09:31,2022-09-09 17:09:31,,,ef013b890f53b98b41611f508554c7084d6b3a28,apache,main,fa40bf92015e307900c2f5c03de8404490b5ef4b,62535543802631a0d3daf0b0b78c6a7e05e508fb,,,,,,0,0,0,0
-github:GithubPullRequest:1:1051273681,2022-09-15 05:36:40.170,2022-09-15 05:36:40.170,"{""ConnectionId"":1,""Owner"":""apache"",""Repo"":""incubator-devlake""}",_raw_github_api_pull_requests,12693,,github:GithubRepo:1:384111310,github:GithubRepo:1:384111310,closed,fix: update pipelineId,"""# Summary\r\n\r\nupdate pipelineId\r\n\r\n### Does this close any open issues?\r\nrelated to #2998 \r\n\r\n### Screenshots\r\n\r\n\r\n### Other Information\r\nAny other information that is important to this PR.\r\n""",https://github.com/apache/incubator-devlake/pull/3034,abeizn,github:GithubAccount:1:101256042,,3034,2022-09-09 08:31:39,2022-09-09 14:19:14,2022-09-09 14:19:14,,,25b90b1d5e8d9430878f90962c9e8f36ee2c49ca,fix#2998-2,main,075e3d3b480e407aede5a0893bd61de9b0fd051d,209168a4b12d91a7ad1f3711fe05b39d5bd8e260,,,,,,0,0,0,0
-github:GithubPullRequest:1:1051340471,2022-09-15 05:36:40.170,2022-09-15 05:36:40.170,"{""ConnectionId"":1,""Owner"":""apache"",""Repo"":""incubator-devlake""}",_raw_github_api_pull_requests,12694,,github:GithubRepo:1:384111310,github:GithubRepo:1:384111310,closed,Jenkins fix,"""# Summary\r\nAdd TestJenkinsStagesDataFlow.\r\nAdd path to stages collect.\r\n\r\n<!--\r\nThanks for submitting a pull request!\r\n\r\nWe appreciate you spending the time to work on these changes.\r\nPlease fill out as many sections below as possible.\r\n-->\r\n\r\n### Does this close any open issues?\r\nCloses #3026 \r\n\r\n### Screenshots\r\n\r\n\r\n\r\n\r\n\r\n### Other Information\r\nAny other information that is important to this PR.\r\n""",https://github.com/apache/incubator-devlake/pul
l/3035,mappjzc,github:GithubAccount:1:2921251,,3035,2022-09-09 09:36:24,2022-09-09 14:47:54,2022-09-09 14:47:54,,,31478092a0afe9bae7de7bdf22e329778564b83f,jenkins-fix,main,075e3d3b480e407aede5a0893bd61de9b0fd051d,08d2f2b6de0fa8de4d0e2b55b4b9a2e244214029,,,,,,0,0,0,0
-github:GithubPullRequest:1:1051524882,2022-09-15 05:36:40.170,2022-09-15 05:36:40.170,"{""ConnectionId"":1,""Owner"":""apache"",""Repo"":""incubator-devlake""}",_raw_github_api_pull_requests,12696,,github:GithubRepo:1:384111310,github:GithubRepo:1:384111310,closed,fix: task failure caused by 404,"""# Summary\r\nfix #2960 [Bug][gitihub] collect account failed by not found user\r\nIn the case of `err == ErrIgnoreAndContinue`, the `err` should not be wrapped, because on the caller side, the expression `err == ErrIgnoreAndContinue` would evaluate as false.\r\n```go\r\nif apiClient.afterResponse != nil {\r\n\t\terr = apiClient.afterResponse(res)\r\n\t\tif err != nil && err != ErrIgnoreAndContinue {\r\n\t\t\tres.Body.Close()\r\n\t\t\treturn nil, errors.Default.Wrap(err, fmt.Sprintf(\""error running afterRequest for %s\"", req.URL.String()), errors.UserMessage(\""error after making API call\""))\r\n\t\t}\r\n\t}\r\n```\r\n### Does this close any open issues?\r\nCloses #2960 \r\n\r\n### Scr
eenshots\r\nInclude any relevant screenshots here.\r\n\r\n### Other Information\r\nAny other information that is important to this PR.\r\n""",https://github.com/apache/incubator-devlake/pull/3038,mindlesscloud,github:GithubAccount:1:8455907,,3038,2022-09-09 12:35:29,2022-09-09 13:32:51,2022-09-09 13:32:51,,,9dc48f963e4f642d385bfe03a60c5d0e33424cb6,ignore-and-continue,main,075e3d3b480e407aede5a0893bd61de9b0fd051d,56b895f0443730c6d7abfbc51a05ab35abd2971f,,,,,,0,0,0,0
-github:GithubPullRequest:1:1051574863,2022-09-15 05:36:40.170,2022-09-15 05:36:40.170,"{""ConnectionId"":1,""Owner"":""apache"",""Repo"":""incubator-devlake""}",_raw_github_api_pull_requests,12697,,github:GithubRepo:1:384111310,github:GithubRepo:1:384111310,closed,fix(jenkins): update e2e,"""# Summary\r\n\r\nUpdate e2e according to recent changes\r\n\r\n### Does this close any open issues?\r\nrelate to #2854\r\n\r\n\r\n### Screenshots\r\nInclude any relevant screenshots here.\r\n\r\n### Other Information\r\nAny other information that is important to this PR.\r\n""",https://github.com/apache/incubator-devlake/pull/3039,warren830,github:GithubAccount:1:39366025,,3039,2022-09-09 13:23:37,2022-09-09 15:12:23,2022-09-09 15:12:23,,,ac0be9ea3a0de2f50246907d33eecb46eaf269ea,fix-update-jenkins-e2e,main,31478092a0afe9bae7de7bdf22e329778564b83f,6fd8efd6b2154da8a0ecf9770d2ac23c8f3111b6,,,,,,0,0,0,0
-github:GithubPullRequest:1:1051637383,2022-09-15 05:36:40.170,2022-09-15 05:36:40.170,"{""ConnectionId"":1,""Owner"":""apache"",""Repo"":""incubator-devlake""}",_raw_github_api_pull_requests,12699,,github:GithubRepo:1:384111310,github:GithubRepo:1:384111310,closed,fix(gitlab): update e2e,"""# Summary\r\n\r\nupdate gitlab e2e according to recent changes\r\n\r\n### Does this close any open issues?\r\nrelate to #2871\r\n\r\n\r\n### Screenshots\r\nInclude any relevant screenshots here.\r\n\r\n### Other Information\r\nAny other information that is important to this PR.\r\n""",https://github.com/apache/incubator-devlake/pull/3041,warren830,github:GithubAccount:1:39366025,,3041,2022-09-09 14:17:24,2022-09-09 15:16:28,2022-09-09 15:16:28,,,fa40bf92015e307900c2f5c03de8404490b5ef4b,fix-update-gitlab-e2e,main,ac0be9ea3a0de2f50246907d33eecb46eaf269ea,9d53fb594958e65456793caa1bfa8d07a7614291,,,,,,0,0,0,0
+id,created_at,updated_at,_raw_data_params,_raw_data_table,_raw_data_id,_raw_data_remark,base_repo_id,head_repo_id,status,title,description,url,author_name,author_id,parent_pr_id,pull_request_key,created_date,merged_date,closed_date,type,component,merge_commit_sha,head_ref,base_ref,base_commit_sha,head_commit_sha
Review Comment:
The diversity of the table is far from enough, please add pr that belongs to other repositories, pr with different statuses, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org