You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@devlake.apache.org by "martynvdijke (via GitHub)" <gi...@apache.org> on 2025/02/12 01:17:38 UTC
[I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
martynvdijke opened a new issue, #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
## Question
Hi all,
I have a question I hope you can help me with.
I see from the roadmap over https://devlake.apache.org/docs/Overview/Roadmap/ that Code Quality / Testing is on the roadmap.
Could you maybe explain these data models a bit more and how you plan on getting / ingesting this data?
And your vision and idea behind that is, what you want to measure and how ?
Background:
I want to see if it is possible to measure code coverage and quality metrics next to build time and correlate them.
We have some long ci build pipelines and are curious if we can fine tune the number of tests vs times, or at least gain insights in them at best and maybe split them up or so.
We have the feeling this can be better in balance at least.
I initially thought of using the webhook for this, but I now found out that the webhooks really are hooked into data models (which makes sense).
I am not sure if you are thinking about adding a generic webook data model, so that special use cases can or extra data can be added ?
So long story short, can you maybe explain the vision and idea behind the Code Quality / Testing data model ?
Also awsome work on this project so far !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2817411559
This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "martynvdijke (via GitHub)" <gi...@apache.org>.
martynvdijke commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2345424599
HI @d4x1, sorry it took a bit longer then expected.
I tried to follow your guidelines but I cannot really make it work.
Or cannot really understand the entire flow of how it should fit together and program this in go.
Would it be possible for you to pickup ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "martynvdijke (via GitHub)" <gi...@apache.org>.
martynvdijke commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2250462308
Hey,
I think option 1 would be the nicest it would allow you to query the data in grafana in the best way I think.
Or at least from a user perspective in the best way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2800209574
This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2238566778
<!-- Greeting -->
Hey @martynvdijke!
I'm a bot here to help you with bugs, questions, and contributions. Let's tackle that issue together!
<!-- Answer -->
The Code Quality / Testing data model in Apache DevLake includes a comprehensive set of metrics and data points designed to provide detailed insights into code quality and testing. Here are the specific metrics included:
- `ProjectKey`
- `FileName`
- `FilePath`
- `FileLanguage`
- `CodeSmells`
- `SqaleIndex`
- `SqaleRating`
- `Bugs`
- `ReliabilityRating`
- `Vulnerabilities`
- `SecurityRating`
- `SecurityHotspots`
- `SecurityHotspotsReviewed`
- `SecurityReviewRating`
- `Ncloc`
- `UncoveredLines`
- `Coverage`
- `LinesToCover`
- `DuplicatedLinesDensity`
- `DuplicatedBlocks`
- `DuplicatedFiles`
- `DuplicatedLines`
- `EffortToReachMaintainabilityRatingA`
- `Complexity`
- `CognitiveComplexity`
- `NumOfLines`
These metrics are defined in the `CqFileMetrics` struct in the `cq_file_metrics.go` file <sup>[[1]](https://github.com/apache/incubator-devlake/blob/main/backend/core/models/domainlayer/codequality/cq_file_metrics.go)</sup>.
### Ingesting Data
Apache DevLake ingests data related to code quality and testing from various DevOps tools through its integrative DevOps data plugins. These plugins include tools like GitHub, GitLab, JIRA, Jenkins, and others. The data collection, expansion, and conversion to the domain layer are managed by running subtasks within these plugins. Users can trigger these tasks via a configuration UI or API <sup>[[2]](https://github.com/apache/incubator-devlake-website/blob/main/blog/2022-06-17-How DevLake is up and running/index.md)</sup>.
### Vision and Idea
The vision behind the Code Quality / Testing data model is to provide a unified platform that can ingest, analyze, and visualize fragmented data from various DevOps tools. This allows teams to extract actionable insights for engineering excellence, developer experience, and community growth. By correlating code quality metrics with other data points like build times, teams can optimize their development processes, fine-tune CI pipelines, and balance the number of tests versus build times.
### Webhooks for Custom Data
Regarding the use of webhooks for custom or special use case data, Apache DevLake currently supports pushing data such as "incidents" and "deployments" required by DORA metrics. While the webhook system is quite flexible, it is currently limited to specific data types. However, the system can be configured to relate incidents with deployments at different levels (e.g., service level, product level) <sup>[[3]](https://github.com/apache/incubator-devlake-website/blob/main/versioned_docs/version-v0.21/Plugins/webhook.md)</sup>.
### Correlating Metrics
Apache DevLake correlates different types of data by using a standard data model to bring together data from across the Software Development Life Cycle (SDLC). This unified data integration allows for the creation of intuitive, use-case driven dashboards that provide actionable insights. For example, you can measure code coverage and quality metrics alongside build times to identify areas for optimization in your CI pipelines <sup>[[4]](https://github.com/apache/incubator-devlake-website/blob/main/docs/Overview/Introduction.md)</sup>.
If you have any more specific questions or need further details, feel free to ask!
<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "d4x1 (via GitHub)" <gi...@apache.org>.
d4x1 commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2250706995
@martynvdijke Are you interested in submiting a PR? It doesn't need too much work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "d4x1 (via GitHub)" <gi...@apache.org>.
d4x1 commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2247133685
@martynvdijke Thanks for your issue.
How to process Code Quality is a difficult question.
I cannot answer your question at once, but here are some information and my ideas I can provide.
We have only one datasource SonarQube, that belongs to `DOMAIN_TYPE_CODE_QUALITY` at present, and the domain layer tables which as be found https://github.com/apache/incubator-devlake/tree/main/backend/core/models/domainlayer/codequality are mapped from SonarQube's data models directly.
As for your questions,
1. "I am not sure if you are thinking about adding a generic webook data model, so that special use cases can or extra data can be added ?"
You can describe it in detail, adding a webhook is easy, but what do you mean by `adding a GENRIC webhook data model`?
2. "Fine tune the number of tests vs times"
You can
a. try to build a relation: incident -> line of code -> test, tests with a high incident rate should be run as much as possible.
b. calculate the overlap of tests, and mark some tests are more efficient.
We haven't received enough feedbacks about code quality, so you can give us some advice on it. We can explore it together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "martynvdijke (via GitHub)" <gi...@apache.org>.
martynvdijke commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2247807601
Thanks for the response this clears parts up, thanks for taking the time 👍
1. My tought is that a generic data model / data source could be a nice addition to allow for some customization or realted uses cases that are currently not fully covered. A use case now would be if we can use a generic webhook data model to upload our python code coverage reports. So that we can use that in a custom dashboard.
2. That sounds like a nice plan, will do some tinkering around to see whats possible
We are also searching for a bit here, but I think high over we would want to at least start with:
1. Upload code coverage metrics of our unit and feature tests (in python)
2. Upload the code complexity score
So that we can track that over time and see our improvments in this area.
Hope this makes a bit more clear ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "d4x1 (via GitHub)" <gi...@apache.org>.
d4x1 commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2258063659
@martynvdijke
1. Start a new api handler in plugin webhook
2. In this handler
1. get request body
2. parse request body
3. add new tables or new fields in a certain table according to the request data (I think this will help: https://github.com/apache/incubator-devlake/blob/main/backend/plugins/customize/service/service.go#L113)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #7765: [Question][Data Models] Code Quality / Testing
URL: https://github.com/apache/incubator-devlake/issues/7765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "martynvdijke (via GitHub)" <gi...@apache.org>.
martynvdijke commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2253354228
Hey,
Yeah I should have some time next week to dive into this and submit a PR for this.
If you have time some pointers on how to get the database formatting from the json would be nice, never done anything like that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2481701801
This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #7765: [Question][Data Models] Code Quality / Testing
URL: https://github.com/apache/incubator-devlake/issues/7765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "martynvdijke (via GitHub)" <gi...@apache.org>.
martynvdijke commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2345501176
OKwey I have some code on https://github.com/martynvdijke/devlake/tree/feat-generic-webhook
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "d4x1 (via GitHub)" <gi...@apache.org>.
d4x1 commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2357384431
@martynvdijke Let me see if I can help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "d4x1 (via GitHub)" <gi...@apache.org>.
d4x1 commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2249319939
@martynvdijke
Webhook is suitable for posting some additional data.
Let's talk about this `generic data model` first, it can be added with little effort.
If you post a json data, what is the expected effect? A new table is created and all json fields are flattened and have a corresponding column? Or just set a new table like:
| id | data |
| -------- | ------- |
| x | {json_data} |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Question][Data Models] Code Quality / Testing [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7765:
URL: https://github.com/apache/incubator-devlake/issues/7765#issuecomment-2614654399
This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org