You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by "robgutsopedra (via GitHub)" <gi...@apache.org> on 2023/04/10 10:48:32 UTC

[GitHub] [incubator-devlake] robgutsopedra opened a new issue, #4889: [Feature][Grafana] Workflow metrics

robgutsopedra opened a new issue, #4889:
URL: https://github.com/apache/incubator-devlake/issues/4889

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Use case
   
   As a platform engineer, I'd love to have some data regarding workflow metrics - how much time do my workflows take to execute, which workflows are the slowest, how many workflows end up in failure... 
   
   ### Description
   
   I have been working on some personal metrics to check if updating a machine type of self-hosted runners had a real impact on the workflow timings - and at some point I realized devLake would benefit from having this kind of useful data. 
   Did my new cache implementation _actually_ improved the performance of this workflow?
   Which are our slowest workflows - what makes them difference from the rest? 
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] Startrekzky commented on issue #4889: [Feature][Grafana] Workflow metrics

Posted by "Startrekzky (via GitHub)" <gi...@apache.org>.
Startrekzky commented on issue #4889:
URL: https://github.com/apache/incubator-devlake/issues/4889#issuecomment-1514196549

   Hi @robgutsopedra , is there anything I can help?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] Startrekzky closed issue #4889: [Feature][Grafana] Workflow metrics

Posted by "Startrekzky (via GitHub)" <gi...@apache.org>.
Startrekzky closed issue #4889: [Feature][Grafana] Workflow metrics 
URL: https://github.com/apache/incubator-devlake/issues/4889


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] Startrekzky commented on issue #4889: [Feature][Grafana] Workflow metrics

Posted by "Startrekzky (via GitHub)" <gi...@apache.org>.
Startrekzky commented on issue #4889:
URL: https://github.com/apache/incubator-devlake/issues/4889#issuecomment-1502960937

   It seems you queried workflow metrics from table.cicd_tasks. 
   ![image](https://user-images.githubusercontent.com/14050754/231111508-857ce32b-2edf-4c12-b1e5-5cbdebc3c62c.png)
   
   Given that a DevLake cicd_task is the equivalent of a GitHub action job under a GitHub workflow run, I'm guessing what you actually want to measure is the duration of jobs, not workflow runs. Did I get it right?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] robgutsopedra commented on issue #4889: [Feature][Grafana] Workflow metrics

Posted by "robgutsopedra (via GitHub)" <gi...@apache.org>.
robgutsopedra commented on issue #4889:
URL: https://github.com/apache/incubator-devlake/issues/4889#issuecomment-1502854760

   1. Yes! I'm afraid those are the ones I have more familiarity with. I have tested everything using github_actions. I'm guessing it _should_ work for any workflow system supported by devLake, as the tables should be the same, but can't say for sure!
   2. Of course!
   ![image](https://user-images.githubusercontent.com/101654801/231092818-ea2f4c2b-d557-44ec-a3d7-116a8dad1c4c.png)
   
   I wanted to use the "month" filter a little bit more efficiently, but I'm still playing around - you can ignore it for the moment


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] Startrekzky commented on issue #4889: [Feature][Grafana] Workflow metrics

Posted by "Startrekzky (via GitHub)" <gi...@apache.org>.
Startrekzky commented on issue #4889:
URL: https://github.com/apache/incubator-devlake/issues/4889#issuecomment-1548009745

   Also, are you interested in sharing the dashboard (put up a PR) after you've polished the dashboard and panel names and etc.? I found it really useful to other users as well.
   
   https://user-images.githubusercontent.com/101654801/231092818-ea2f4c2b-d557-44ec-a3d7-116a8dad1c4c.png
   
   Please follow the steps to finish your contribution:
   1. **Back up** the customized Grafana dashboards before upgrading. 
   2. Upgrade to the up-to-date version [v0.17.0-beta5](https://github.com/apache/incubator-devlake/releases/tag/v0.17.0-beta5), see the [upgrade doc](https://devlake.apache.org/docs/GettingStarted/DockerComposeSetup#upgrade-to-a-newer-version). The reason for upgrade is that DevLake has some breaking changes in the [CICD-related entities](https://devlake.apache.org/docs/DataModels/DevLakeDomainLayerSchema#domain-4---cicd) and the definition of `deployments` in v0.17. 
   3. Re-import the dashboard to DevLake, following the screenshots in #4410 , and keep implementing the dashboard according to the latest data models.
   4. Put up a PR for the dashboard.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] Startrekzky commented on issue #4889: [Feature][Grafana] Workflow metrics

Posted by "Startrekzky (via GitHub)" <gi...@apache.org>.
Startrekzky commented on issue #4889:
URL: https://github.com/apache/incubator-devlake/issues/4889#issuecomment-1502621000

   Hi @robgutsopedra , 
   1. I noticed that your workflow metrics are about CICD workflows, is that correct? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] robgutsopedra commented on issue #4889: [Feature][Grafana] Workflow metrics

Posted by "robgutsopedra (via GitHub)" <gi...@apache.org>.
robgutsopedra commented on issue #4889:
URL: https://github.com/apache/incubator-devlake/issues/4889#issuecomment-1501670212

   `{
       "annotations": {
         "list": [
           {
             "builtIn": 1,
             "datasource": "-- Grafana --",
             "enable": true,
             "hide": true,
             "iconColor": "rgba(0, 211, 255, 1)",
             "name": "Annotations & Alerts",
             "type": "dashboard"
           }
         ]
       },
       "editable": true,
       "gnetId": null,
       "graphTooltip": 0,
       "id": 28,
       "iteration": 1681123370748,
       "links": [],
       "panels": [
         {
           "datasource": null,
           "gridPos": {
             "h": 3,
             "w": 24,
             "x": 0,
             "y": 0
           },
           "id": 8,
           "options": {
             "content": "- Use Cases: This dashboard presents some metrics about your workflows. \n- Data Source Required: GitHub \n- This panel is still being built",
             "mode": "markdown"
           },
           "pluginVersion": "8.0.6",
           "targets": [
             {
               "queryType": "randomWalk",
               "refId": "A"
             }
           ],
           "title": "Dashboard Introduction",
           "type": "text"
         },
         {
           "collapsed": false,
           "datasource": null,
           "gridPos": {
             "h": 1,
             "w": 24,
             "x": 0,
             "y": 3
           },
           "id": 6,
           "panels": [],
           "title": "Workflow metrics",
           "type": "row"
         },
         {
           "datasource": "mysql",
           "fieldConfig": {
             "defaults": {
               "color": {
                 "mode": "palette-classic"
               },
               "custom": {
                 "axisLabel": "",
                 "axisPlacement": "auto",
                 "barAlignment": 0,
                 "drawStyle": "line",
                 "fillOpacity": 0,
                 "gradientMode": "none",
                 "hideFrom": {
                   "legend": false,
                   "tooltip": false,
                   "viz": false
                 },
                 "lineInterpolation": "linear",
                 "lineWidth": 1,
                 "pointSize": 5,
                 "scaleDistribution": {
                   "type": "linear"
                 },
                 "showPoints": "auto",
                 "spanNulls": false,
                 "stacking": {
                   "group": "A",
                   "mode": "none"
                 },
                 "thresholdsStyle": {
                   "mode": "off"
                 }
               },
               "mappings": [],
               "thresholds": {
                 "mode": "absolute",
                 "steps": [
                   {
                     "color": "green",
                     "value": null
                   },
                   {
                     "color": "red",
                     "value": 80
                   }
                 ]
               }
             },
             "overrides": []
           },
           "gridPos": {
             "h": 9,
             "w": 24,
             "x": 0,
             "y": 4
           },
           "id": 2,
           "options": {
             "legend": {
               "calcs": [],
               "displayMode": "list",
               "placement": "bottom",
               "showLegend": true
             },
             "tooltip": {
               "mode": "single",
               "sort": "none"
             }
           },
           "targets": [
             {
               "format": "time_series",
               "group": [],
               "hide": false,
               "metricColumn": "none",
               "rawQuery": true,
               "rawSql": "SELECT \n  DATE_ADD(date(cicd.started_date), INTERVAL -DAY(date(cicd.started_date))+1 DAY) as time,\n  AVG(duration_sec) as avg_duration\nFROM \n  cicd_tasks as cicd\n  join project_mapping pm on cicd.cicd_scope_id = pm.row_id\nWHERE \n  duration_sec < 10000 and \n  pm.project_name in ($project) and\n  $__timeFilter(cicd.started_date)\nGROUP BY time\norder by time",
               "refId": "A",
               "select": [
                 [
                   {
                     "params": [
                       "blueprint_id"
                     ],
                     "type": "column"
                   }
                 ]
               ],
               "table": "_devlake_blueprint_labels",
               "timeColumn": "created_at",
               "timeColumnType": "timestamp",
               "where": [
                 {
                   "name": "$__timeFilter",
                   "params": [],
                   "type": "macro"
                 }
               ]
             }
           ],
           "title": "Average duration of workflows through time",
           "type": "timeseries"
         },
         {
           "datasource": "mysql",
           "fieldConfig": {
             "defaults": {
               "color": {
                 "mode": "thresholds"
               },
               "custom": {
                 "align": "auto",
                 "displayMode": "auto"
               },
               "mappings": [],
               "thresholds": {
                 "mode": "absolute",
                 "steps": [
                   {
                     "color": "green",
                     "value": null
                   },
                   {
                     "color": "red",
                     "value": 80
                   }
                 ]
               },
               "unit": "s"
             },
             "overrides": []
           },
           "gridPos": {
             "h": 8,
             "w": 24,
             "x": 0,
             "y": 13
           },
           "id": 4,
           "options": {
             "footer": {
               "countRows": false,
               "enablePagination": false,
               "fields": "",
               "reducer": [
                 "sum"
               ],
               "show": false
             },
             "showHeader": true
           },
           "pluginVersion": "8.0.6",
           "targets": [
             {
               "datasource": "mysql",
               "editorMode": "code",
               "format": "table",
               "rawQuery": true,
               "rawSql": "SELECT \n  cs.name as repo_name,\n  cicd.name as workflow_name,\n  AVG(duration_sec) as avg_duration\nFROM lake.cicd_tasks as cicd\n  JOIN cicd_scopes cs \non cs.id = cicd_scope_id\n  join project_mapping pm on cicd.cicd_scope_id = pm.row_id \nWHERE \n  duration_sec < 10000 AND \n  pm.project_name in ($project)\nGROUP by repo_name,workflow_name\nORDER BY avg_duration DESC\nLIMIT 40",
               "refId": "A",
               "sql": {
                 "columns": [
                   {
                     "parameters": [],
                     "type": "function"
                   }
                 ],
                 "groupBy": [
                   {
                     "property": {
                       "type": "string"
                     },
                     "type": "groupBy"
                   }
                 ],
                 "limit": 50
               }
             }
           ],
           "title": "Top 40 slowest workflows",
           "type": "table"
         }
       ],
       "refresh": false,
       "revision": 1,
       "schemaVersion": 30,
       "style": "dark",
       "tags": [],
       "templating": {
         "list": [
           {
             "allValue": null,
             "current": {
               "selected": true,
               "text": [
                 "ultimateai"
               ],
               "value": [
                 "ultimateai"
               ]
             },
             "datasource": "mysql",
             "definition": "select distinct name from projects",
             "description": null,
             "error": null,
             "hide": 0,
             "includeAll": true,
             "label": "Project",
             "multi": true,
             "name": "project",
             "options": [],
             "query": "select distinct name from projects",
             "refresh": 1,
             "regex": "",
             "skipUrlSync": false,
             "sort": 0,
             "type": "query"
           },
           {
             "allValue": null,
             "current": {
               "selected": false,
               "text": "2023-04",
               "value": "2023-04-01"
             },
             "datasource": "mysql",
             "definition": "select\n  distinct(concat(date_format(DATE_ADD(date(created_date), INTERVAL -DAY(date(created_date))+1 DAY), '%Y-%m') , ':', date_format(DATE_ADD(date(created_date), INTERVAL -DAY(date(created_date))+1 DAY), '%Y-%m-%d'))) as month\nfrom\n  lake.issues i\norder by month desc",
             "description": null,
             "error": null,
             "hide": 0,
             "includeAll": false,
             "label": "Month",
             "multi": false,
             "name": "month",
             "options": [],
             "query": "select\n  distinct(concat(date_format(DATE_ADD(date(created_date), INTERVAL -DAY(date(created_date))+1 DAY), '%Y-%m') , ':', date_format(DATE_ADD(date(created_date), INTERVAL -DAY(date(created_date))+1 DAY), '%Y-%m-%d'))) as month\nfrom\n  lake.issues i\norder by month desc",
             "refresh": 1,
             "regex": "/^(?<text>.*):(?<value>.*)$/",
             "skipUrlSync": false,
             "sort": 0,
             "type": "query"
           }
         ]
       },
       "time": {
         "from": "now-6M",
         "to": "now"
       },
       "timepicker": {},
       "timezone": "",
       "title": "WIP - workflow metrics",
       "uid": "K8VkM8Y4k",
       "version": 4
     }`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] robgutsopedra commented on issue #4889: [Feature][Grafana] Workflow metrics

Posted by "robgutsopedra (via GitHub)" <gi...@apache.org>.
robgutsopedra commented on issue #4889:
URL: https://github.com/apache/incubator-devlake/issues/4889#issuecomment-1502973297

   yes! I'll have a look
   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] Startrekzky commented on issue #4889: [Feature][Grafana] Workflow metrics

Posted by "Startrekzky (via GitHub)" <gi...@apache.org>.
Startrekzky commented on issue #4889:
URL: https://github.com/apache/incubator-devlake/issues/4889#issuecomment-1547496539

   I'll close it and feel free to reopen it if this issue still exists.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] robgutsopedra commented on issue #4889: [Feature][Grafana] Workflow metrics

Posted by "robgutsopedra (via GitHub)" <gi...@apache.org>.
robgutsopedra commented on issue #4889:
URL: https://github.com/apache/incubator-devlake/issues/4889#issuecomment-1519089868

   Sorry for the huge delay, @Startrekzky ! Holidays+crazy week =(
   
   Yeah, I was confused due to reusable workflows naming (caller_name/callable_name) but you're right, these metrics are for jobs (which makes sense, if a workflow run is composed of three jobs: 1 minute + 1 minute + 20 minutes, I'm more interested in the job taking 20 minutes, than the workflow run taking 22) 
   
   I'll update quickly a couple of names from workflow to job


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org