[GitHub] [incubator-devlake] klesh opened a new issue, #4534: [Feature][infra] Blueprint/Pipeline Advanced Mode Supports Project/DORA

["klesh (via GitHub)" <gi...@apache.org>]

klesh opened a new issue, #4534:
URL: https://github.com/apache/incubator-devlake/issues/4534

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Use case
   
   ### Why
   
   DORA is no doubt one of the most important features of our system judged by the feedback from our new users, many of them joined the community because of it.
   
   In our current design (v0.15), to calculate DORA metrics, we need `project_mapping` to be populated before the DORA plugin gets executed. However, this operation happens during `blueprint.settings` to `blueprint.plan` conversion which is only available in Normal Mode. In other words, Advanced Mode accepts `blueprint.plan` directly **without** conversion, thus no DORA for Advanced Mode. It poses a couple of problems:
   
   1.  Not all our plugins support Normal Mode, especially those in WIP/Beta stage like BitBucket/TaPD/Zentao. It is a blocker for users who are using these Data Sources and are willing to help with testing.
   2.  Some users might want to arrange their collection/computation in a particular way. i.e. to collect data in one blueprint but compute the DORA metric in another one. However, Normal Mode has a different agenda(guiding new users), which makes Advanced Mode the only suitable way to support this kind of customization.
   
   In conclusion, **Advanced Mode must support Project/DORA.**
   
   ### Description
   
   ### What
   
   The DORA plugin itself merely depends on the `project_mapping` table which is OK, however, the table is managed by the `framework` and omitted from the `bp.plan` which is the root cause of the problem. I hereby proposed the following solution to fix it without too much hassle:
   
   **Let the plugin maintains the `project_mapping` instead of the `framework` so it can be described in the `bp.plan` field**, for example, the `org` plugin:
   
   Pipeline task json example:
   
   ```json
   {
       "plugin": "org",
       "subtasks": [ "setProjectMapping" ],
       "options": {
           "projectMappings": [{
               "projectName": "myproject",
               "scopes": [
                   { "table": "cicd_scope", "row_id": "gitlab:GitlabProject:1:11876178" },
                   { "table": "boards", "row_id": "gitlab:GitlabProject:1:11876178" }
               ]
           }]
       }
   }
   ```
   
   Meanwhile, the Normal Mode logic must be updated to produce the `task` instead of writing the table directly.
   
   ### How
   
   With this solution, the Project relationship can be fully described in the `bp.plan` field, it offers more freedom to users to do whatever they see fit.
   
   1.  One might test/use the Beta/WIP plugins in Advanced Mode. To that end, we must provide thorough documentation to guide users.
     a.  Documents to describe `scopes` management for WIP/Beta plugins.
     b.  Add a section to the Project Doc to describe how the `plan` works and where the `org task` should be.
     c.  Offer a workable example `plan` with related docs linked.
   2.  For those who want collection/computation in separate blueprints, we shall offer a document with examples to guide them through the process.
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-devlake] warren830 closed issue #4534: [Feature][infra] Blueprint/Pipeline Advanced Mode Supports Project/DORA

["warren830 (via GitHub)" <gi...@apache.org>]

warren830 closed issue #4534: [Feature][infra] Blueprint/Pipeline Advanced Mode Supports Project/DORA
URL: https://github.com/apache/incubator-devlake/issues/4534


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org