You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@devlake.apache.org by "narrowizard (via GitHub)" <gi...@apache.org> on 2025/04/21 00:59:50 UTC
[I] [Feature][Domain layer] Add QA Domain Support and Enable CSV Import into QA Domain via Customize Plugin [incubator-devlake]
narrowizard opened a new issue, #8393:
URL: https://github.com/apache/incubator-devlake/issues/8393
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement.
### Use case
As a DevOps or engineering insights team, we want to analyze and visualize testing-related data (such as test cases, test runs, test results, and test coverage) as part of our software development lifecycle metrics. To do this, we need a structured QA domain within DevLake’s domain layer. Additionally, we want to import test data from custom or third-party sources in CSV format using the Customize plugin, allowing us to map CSV fields directly into the new QA domain entities.
### Description
Currently, DevLake supports multiple domains such as issue tracking, code management, CI/CD, etc., abstracting data from various tools into a unified domain layer. However, there is no dedicated QA domain for test-related data. This limits our ability to track test activity and quality metrics across the development lifecycle.
We propose the following enhancements:
1. Introduce a QA Domain Layer
- Define a set of QA-related domain entities (e.g., TestCase, TestRun, TestResult, TestSuite, etc.).
- These entities should be designed as abstractions that can map to various testing tools or custom sources (e.g., from Jira test cases, Zephyr, Xray, TestRail, etc.).
2. Enhance the Customize Plugin
- Enable CSV import functionality for the new QA domain.
- Ensure this includes support for relationships between test entities (e.g., a TestResult belonging to a specific TestRun).
This feature would allow teams to bring in fragmented QA/test data from external tools and integrate it into DevLake’s unified model for analysis and visualization.
### Related issues
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature][Domain layer] Add QA Domain Support and Enable CSV Import into QA Domain via Customize Plugin [incubator-devlake]
Posted by "narrowizard (via GitHub)" <gi...@apache.org>.
narrowizard commented on issue #8393:
URL: https://github.com/apache/incubator-devlake/issues/8393#issuecomment-2817449196
**Implementation Plan**
To support a new QA domain in DevLake and enable CSV import through the Customize plugin, the following implementation plan is proposed:
1. **QA Domain Schema Design**
Introduce a dedicated QA domain to capture testing-related data. Core entities:
- `QaProject`: Represents a testing project or scope. It will be implicitly created via test case data.
- `QaApi` *(optional)* – represents API metadata for coverage metrics.
- `QaTestCase` – represents individual test cases, including type (functional or API), and links to APIs and projects.
- `QaTestCaseExecution` – represents execution records of test cases, including timestamps and status.
`QaProject` will be derived from the `qa_test_cases.csv` import, using fields like `qa_project_id` and `qa_project_name`.
2. **Customize Plugin Extension**
Extend the Customize plugin to support CSV imports for the QA domain using standardized endpoints. Each CSV file corresponds to one importable entity via the following pattern:
| CSV File | API Endpoint | Description |
|--------------------------------|--------------------------------------------------------------|--------------------------------|
| `qa_apis.csv` *(optional)* | `POST /plugins/customize/csvfiles/qa_apis.csv` | Imports API metadata |
| `qa_test_cases.csv` | `POST /plugins/customize/csvfiles/qa_test_cases.csv` | Imports test cases + projects + apis |
| `qa_test_case_executions.csv` | `POST /plugins/customize/csvfiles/qa_test_case_executions.csv` | Imports test execution records |
Each endpoint:
- Accepts a CSV file upload.
- Automatically maps and ingests data into the corresponding QA domain entity.
3. **Incremental Import Support**
- Use `id` as the primary key to identify each row.
- Optional deduplication logic may be applied depending on import mode.
- For `QaProject`, ensure uniqueness via `qa_project_id`.
4. **CSV Format Specification**
Based on MeterSphere schema and export capabilities, the CSV schemas are:
**qa_test_cases.csv**
- `id`, `name`, `create_time`, `creator_id`, `type` (`functional` or `api`)
- `api_id` *(nullable)*, `api_name`, `api_create_time`, `api_creator_id` *(optional, used to create/import `QaApi`)*
- `qa_project_id`, `qa_project_name` *(used to create/import `QaProject`)*
**qa_test_case_executions.csv**
- `id`, `qa_test_case_id`, `qa_project_id`
- `create_time`, `start_time`, `finish_time`
- `creator_id`, `status` (`PENDING`, `IN_PROGRESS`, `SUCCESS`, `FAILED`)
**qa_apis.csv** *(optional)*
- `id`, `name`, `create_time`, `creator_id`, `qa_project_id`
5. **Documentation**
- Add CSV schema definitions and examples to DevLake documentation.
- Provide import instructions for each endpoint using `/plugins/customize/csvfiles/{csvfilename}.csv`.
- Clarify the expected relationships between test cases, executions, APIs, and projects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature][Domain layer] Add QA Domain Support and Enable CSV Import into QA Domain via Customize Plugin [incubator-devlake]
Posted by "Startrekzky (via GitHub)" <gi...@apache.org>.
Startrekzky commented on issue #8393:
URL: https://github.com/apache/incubator-devlake/issues/8393#issuecomment-2833335306
@narrowizard Do we need to add a `batch` to entity QaTestCaseExecution? I think some users would like to measure executions by batch.
For instance,
- batch_1 executes case 1&2&3
- batch_2 executes case 1&5&6
If I were the end user, I would like to see the execution results by batch, such as the failed rate and failed reasons in the last executed batch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature][Domain layer] Add QA Domain Support and Enable CSV Import into QA Domain via Customize Plugin [incubator-devlake]
Posted by "narrowizard (via GitHub)" <gi...@apache.org>.
narrowizard commented on issue #8393:
URL: https://github.com/apache/incubator-devlake/issues/8393#issuecomment-2829139746
Further updates and clarifications on the implementation plan:
1. **Align Creator Field with Issue #8400 and Retain Account Creation:**
* To maintain consistency with the processing logic discussed or implemented in issue #8400, the relevant field for creators in the domain layer entities processed by this customize import (e.g., `qa_apis` or other related tables) should be named `creator_name` instead of `creator_id`.
* **Crucially, the process of automatically creating/linking entries in the `accounts` table based on this `creator_name` during the data import (as discussed in #8400) should still be implemented.** The goal remains to ensure that users referenced by name in the CSV have corresponding entries in the `accounts` table.
2. **Separate Import for QA APIs Data:**
* When importing data specifically through the `qa_api_test_cases.csv` interface within the customize plugin, the process should **no longer include** the automatic creation of `qa_apis` domain data.
* If `qa_apis` data needs to be imported, it should be done through the dedicated `qa_apis.csv` interface provided by the customize plugin. This ensures a clear separation of concerns and data types, promoting better modularity and maintainability of the import interfaces.
These updates refine the plan to align with existing patterns in the codebase and improve the structure of the import interfaces for different data entities.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature][Domain layer] Add QA Domain Support and Enable CSV Import into QA Domain via Customize Plugin [incubator-devlake]
Posted by "narrowizard (via GitHub)" <gi...@apache.org>.
narrowizard commented on issue #8393:
URL: https://github.com/apache/incubator-devlake/issues/8393#issuecomment-2833770502
@Startrekzky Thank you very much for your feedback!
Our original requirement does not include such a detailed perspective.
To keep the functionality simple for this phase, we are not planning to introduce the concept of "batch" at this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature][Domain layer] Add QA Domain Support and Enable CSV Import into QA Domain via Customize Plugin [incubator-devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh closed issue #8393: [Feature][Domain layer] Add QA Domain Support and Enable CSV Import into QA Domain via Customize Plugin
URL: https://github.com/apache/incubator-devlake/issues/8393
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org