You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@devlake.apache.org by "jbsmith7741 (via GitHub)" <gi...@apache.org> on 2026/06/18 16:14:09 UTC
[I] [Feature][SonarQube] Collect historical project-level metrics from SonarQube [devlake]
jbsmith7741 opened a new issue, #8941:
URL: https://github.com/apache/devlake/issues/8941
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement.
### Use case
As an engineering manager, I want to view **historical code coverage and quality
metric trends** for my SonarQube projects so that I can:
- Track whether test coverage is improving or regressing over time.
- Set OKR baselines and measure engineering quality improvements quarter over
quarter.
- Compare quality trajectories across teams and projects.
- Correlate coverage changes with releases and refactors.
Today, DevLake's SonarQube plugin only stores a **point-in-time snapshot** of
metrics. Each sync overwrites the previous values, so there is no way to answer
"How has our coverage trended over the past year?" without an external data
pipeline.
### Description
#### Problem
The SonarQube plugin collects per-file metrics via `measures/component_tree` and
stores them in `_tool_sonarqube_file_metrics` / `cq_file_metrics`. These tables
are keyed by `(connection_id, file_metrics_key)` with no temporal dimension —
every sync replaces the prior row. The only timestamp in the model is
`last_analysis_date` on `_tool_sonarqube_projects` / `cq_projects`, which
reflects only the most recent analysis.
#### SonarQube APIs available but not used
| Endpoint | Purpose |
|----------|---------|
| `api/measures/search_history` | Time-series metric values at project level |
| `api/project_analyses/search` | Analysis metadata (dates, versions, quality gate events) |
`measures/search_history` accepts `from` / `to` date parameters and returns
chronological metric values across all retained analyses for a given project
component. It supports pagination (`ps` max 1000) and the full set of standard
metric keys (`coverage`, `ncloc`, `bugs`, `vulnerabilities`, `code_smells`,
`security_hotspots`, ratings, duplication, complexity, etc.).
#### Why project-level, not file-level
SonarQube's housekeeping [deletes file-level and directory-level measures after
each new analysis](https://community.sonarsource.com/t/api-measures-search-history-returns-only-date-history/730).
Querying `measures/search_history` for a file key returns analysis dates but
**no values**. Project-level history, by contrast, is retained for up to 5 years
(thinned over time by [housekeeping rules](https://docs.sonarsource.com/sonarqube-server/latest/instance-administration/system-functions/housekeeping/)):
| Age | Retention |
|-----|-----------|
| < 1 day | All snapshots |
| > 1 day | 1 snapshot/day |
| > 1 month | 1 snapshot/week |
| > 1 year | 1 snapshot/month |
| > 2 years | Only VERSION-tagged snapshots |
| > 5 years | All deleted |
File-level historical metrics are out of scope for this feature because the
upstream data simply does not exist. The current per-file `cq_file_metrics`
table continues to serve the "latest state" use case.
#### Sync Policy integration
The new collectors should respect the blueprint **Sync Policy `TimeAfter`**
field, following the same `NewStatefulApiCollector` / `CollectorStateManager`
pattern used by the Jira and GitHub plugins:
- **Initial / full sync:** collect from `syncPolicy.TimeAfter` to now.
- **Incremental syncs:** collect from `state.LatestSuccessStart` to now.
- **`fullSync`:** re-fetch the entire history window.
`GetSince()` maps directly to the SonarQube `from` query parameter.
#### Proposed metrics
All returned in a single paginated API call, so there is no added cost for
breadth:
| Category | Metric keys |
|----------|-------------|
| Coverage | `coverage`, `ncloc` |
| Reliability | `bugs`, `reliability_rating` |
| Maintainability | `code_smells`, `sqale_rating`, `complexity`, `cognitive_complexity` |
| Security | `vulnerabilities`, `security_rating`, `security_hotspots` |
| Duplication | `duplicated_lines_density` |
`new_*` variants are intentionally excluded — SonarQube does not retain
reliable history for them.
### Related issues
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature][SonarQube] Collect historical project-level metrics from SonarQube [devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh closed issue #8941: [Feature][SonarQube] Collect historical project-level metrics from SonarQube
URL: https://github.com/apache/devlake/issues/8941
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org