[GitHub] [tvm-rfcs] areusch commented on a diff in pull request #93: Issue Triage Workflow RFC

[GitBox <gi...@apache.org>]

areusch commented on code in PR #93:
URL: https://github.com/apache/tvm-rfcs/pull/93#discussion_r972445408


##########
rfcs/0093_Issue_Triage.md:
##########
@@ -0,0 +1,309 @@
+# Issue Triage
+
+* Feature Name: `Update Issue Triage Workflow`
+* Co-authored with @areusch, @denise
+
+# **Summary**
+
+This RFC proposes expanding functionality to the existing “New Issue” templates in order to allow for more a precise logging and tracking system for GitHub Issues. The updated templates will be leveraging a new `triage-bot` to automatically add relevant labels and cc the appropriate person(s). This RFC also includes establishing more current labels while discontinuing the use outdated labels (see bottom of the RFC for the list).
+
+# **Motivation**
+
+Currently, it is sometimes difficult to find and organize TVM’s GitHub Issues due to the lack of specificity. More specifically, our current means of classifying GitHub Issues is the `label` attribute, and many of the labels help only to identify the kind of issue (bug, feature, docs, etc). Most of our labels only vaguely correspond to logical components of the TVM codebase (for instance: `type: backend` — which backend?) and no longer aligns with the latest dev efforts. It is currently difficult, for example as a BYOC owner, to identify issues that the community believes should be resolved through changes to a BYOC implementation in TVM.
+
+As the codebase grows, there is increasingly a need to be able to organize Issues according to the logical component of the codebase affected. Doing this allows folks with specific expertise and ownership to focus on a smaller set of issues. It also allows us as a community to more concretely establish issue triage as a role, increasing opportunity for advancement through non-code contribution.
+
+GitHub Issue Templates can help with this, but in TVM these are primarily organized around the issue type (bug, feature, docs, etc) rather than sorting an issue by component.
+
+This RFC hopes to more naturally organize TVM’s GitHub Issues by introducing new labels and leveraging a new `triage-bot` which can parse the issue title and body to assign labels and cc the appropriate contributors.
+
+The overall goal of this RFC is to simplify and strengthen the issue triage workflow, in turn allowing the TVM community to more easily navigate and contribute to the project.
+
+# **Guide-level explanation**
+
+## Label Reorganization
+
+Firstly, this RFC proposes to deprecate a number of the labels we currently use to identify codebase components and add a new set. The new set is much larger than the previous set; however, we believe the new set corresponds to the logical structure of TVM with more granularity. As discussed above, we think it’s important that folks volunteering to maintain a component have an easy way to identify the set of tasks/issues that need their attention.
+
+To that end, we suggest the a new set of labels. We also suggest to modify `[CONTRIBUTORS.md](http://CONTRIBUTORS.md)` to reflect each Reviewer and Committer’s expertise in terms of these labels. As future work, the Auto-CC subscription tracker can then automatically better tag those with relevant expertise on relevant issues.
+
+## GitHub Issue Templates
+
+This RFC plans to phase out **`Update CI Docker Image`** going forward as there is no longer a strong need for it. The rest of the templates will stay.
+
+The figure below shows the templates TVM uses now. We suggest to modify all remaining templates to assign a `needs-triage` label by default. This new label will alert either the new `triage-bot` or TVM contributors to sort the issue.
+
+![New Issue Template](assets/0089/New%20Issue%20Templates.png)
+
+When a new issue is created with the `needs-triage` label, the `triage-bot` will parse the issue’s title or text body (see below for a discussion of options) for label tags. If one or more is found, `triage-bot` will remove `needs-triage` and add the found labels to the issue.
+
+This RFC proposes to modify the :bug: **Bug Report** and :wrench: **Feature Tracking** templates to add instructions encouraging users to self-select which component an issue should belong to. These templates will link to a Markdown doc (checked into TVM) that lists all the label tags and includes a small description for the scope of each one. This will help the users find the appropriate label tags to use.
+
+### Example User Workflow
+
+In these templates, the `triage-bot` will parse the issue’s text body for tags in bullet form.
+
+For example, when creating an issue concerning a bug in `autotvm` , the process would be -
+
+1. The user would select the **Bug Report** template and will see a section shown in body to list the tags they would like to use. It will include a link to the document mentioned above which lists all the label tags.
+2. The user would then add the relevant tags. In this case, they would add `autotvm`. In the case that the issue creator is unsure of which label tags to use, they can simply leave the label tag section blank. This template will add the `needs-triage` label by default which can alert other TVM contributors who can assist in properly sorting the issue.
+3. After filling out the quick summary of the issue, they would post it. It will look like shown in this [example here](https://drive.google.com/file/d/12IHgPFcijcSsbHrY5IY9ACqDlpl2AbP9/view?usp=sharing) and will trigger the `triage-bot` to parse the issue body and add the appropriate label.
+4. Then, the `triage-bot` removes the `needs-triage` label and adds the appropriate label. In this example, it would add `autotvm`.
+
+![TVM Bot](assets/0089/TVM%20bot.png)
+
+If `triage-bot` is unable to find any labels, it will leave the issue alone. The `needs-triage` label will continue to indicate issues which need manual triage. We hope by establishing this category that it encourages more community members to work together to triage issues by consolidating those that need it into a single location.
+
+### Rationale
+
+This will improve quality of the issue tracking and make it easier to drill down on specific pain points for different components. Users will be able to quickly sort the issues by their areas of interests and needs.
+
+Issues can have multiple labels tagged, although we ask community members to exercise their judgement and file issues in as few components as the situation necessitates.
+
+Contributors can propose a new label by posting on the Discuss forum. Requests should include adequate scoping to justify the label, as was done in this doc. Labels can similarly be retired via a proposal on Discuss.

Review Comment:
   In general, I think this should be at the discretion of the project--right now, I think that means at the discretion of triagers/committers. For minor extensions of scope (i.e. adding a new file as you say), I think a committer could exercise this discretion independently. For major extensions of scope (i.e. expanding "ir" sub-label to include a new class of IR), this could be posed on the forum.
   
   Additionally, right now there isn't any sort of division of responsibility over the created set of labels. This is covered under Future Work (and in my mind, would be actuated by modfiying CONTRIBUORS.md to state areas of committer/reviewer focus in terms of these labels rather than the somewhat arbitrary set that exist there now). Given all of these factors at play now and that the RFC vote passed, I'd like to suggest we proceed with committer discretion for now. I'd be happy to keep discussing this on a follow-on Discuss thread or RFC PR if that seems problematic to you.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org