You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tvm.apache.org by Denise Kutnick <no...@github.com.INVALID> on 2022/01/20 01:42:05 UTC

[apache/tvm-rfcs] [RFC][Roadmap] TVM Continuous Integration & Testing Roadmap (PR #54)

cc @areusch @leandron @tqchen 
You can view, comment on, or merge this pull request online at:

  https://github.com/apache/tvm-rfcs/pull/54

-- Commit Summary --

  * [RFC][Roadmap] TVM Continuous Integration &amp; Testing Roadmap

-- File Changes --

    A rfcs/0054-ci-testing-roadmap.md (108)

-- Patch Links --

https://github.com/apache/tvm-rfcs/pull/54.patch
https://github.com/apache/tvm-rfcs/pull/54.diff

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/54
You are receiving this because you are subscribed to this thread.

Message ID: &lt;apache/tvm-rfcs/pull/54@github.com&gt;

Re: [apache/tvm-rfcs] [RFC][Roadmap] TVM Continuous Integration & Testing Roadmap (PR #54)

Posted by Christopher Sidebottom <no...@github.com.INVALID>.
Merged #54 into main.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/54#event-6033018851
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>

Re: [apache/tvm-rfcs] [RFC][Roadmap] TVM Continuous Integration & Testing Roadmap (PR #54)

Posted by Andrew Reusch <no...@github.com.INVALID>.
@Mousius One way to group the various security related tasks is like so:
- Infra work: adding e.g. vuln scanners to CI or elsewhere
- Security problems with CI infra: issues outside the TVM codebase, but which are encountered only because we need to run a CI. Things related to e.g. Jenkins version, patching OS base box vulnerabilities, etc. Likely handled by those TVM committers working on CI.
- Security problems with TVM dependencies: issues whose root cause are outside the TVM codebase, but which any TVM user might encounter in the course of using TVM. Ultimately resolved by updating dependencies, but likely requires some collaboration between folks working on TVM core and folks working on making a TVM release (e.g. which would actually specify the dependency versioning requirements and presumably where a future test might run)
- Security vulnerabilities in the TVM code itself: memory overflows and kin, privilege escalation, etc. Resolved just like any other TVM change, but requires a direct resolution in the TVM codebase.

The first two problems, we can tackle without considering how to message those in a release, because they have immediate resolution. The latter two require us to consider how our release process would message these issues, whether we would cherry-pick a workaround/fix to earlier releases, etc. So let's table those two here until we sort out the release process and determine where they'll land then. It's entirely possible we should just create a separate Roadmap for security to track everything centrally, but we're just trying to land this one piece here. We added the security item to address @leandron 's comment, which I think is entirely reasonable, but I think his request was scoped to just the CI infra if I understand it correctly. Would prefer to proceed on this one and consider security holistically in a follow-on RFC.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/54#issuecomment-1031667756
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>

Re: [apache/tvm-rfcs] [RFC][Roadmap] TVM Continuous Integration & Testing Roadmap (PR #54)

Posted by Andrew Reusch <no...@github.com.INVALID>.
@Mousius since @leandron is a bit busy would you like to review?

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/54#issuecomment-1029442133
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>

Re: [apache/tvm-rfcs] [RFC][Roadmap] TVM Continuous Integration & Testing Roadmap (PR #54)

Posted by Denise Kutnick <no...@github.com.INVALID>.
@areusch can you get some additional reviews on this? thanks!

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/54#issuecomment-1026535335
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>

Re: [apache/tvm-rfcs] [RFC][Roadmap] TVM Continuous Integration & Testing Roadmap (PR #54)

Posted by Christopher Sidebottom <no...@github.com.INVALID>.
As an extension of @leandron 's comment above, we should apply the same to packages we install and use as part of TVM - at the bare minimum we should have tools to check packages haven't notified of major vulnerabilities and basic static analysis tools to catch easy to spot vulnerable code. As part of this we need some kind of mechanism for locking versions and performing updates on them regularly (I remember @areusch was keen on `poetry` at one point :wink:).

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/54#issuecomment-1029930458
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>

Re: [apache/tvm-rfcs] [RFC][Roadmap] TVM Continuous Integration & Testing Roadmap (PR #54)

Posted by Andrew Reusch <no...@github.com.INVALID>.
since we're out of cycles to upstream tasks today, @Mousius can you take another look and make sure this agrees with you, and feel free to approve/merge if so.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/54#issuecomment-1030442279
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>

Re: [apache/tvm-rfcs] [RFC][Roadmap] TVM Continuous Integration & Testing Roadmap (PR #54)

Posted by Christopher Sidebottom <no...@github.com.INVALID>.
> @leandron @Mousius thanks for taking a look! @denise-k updated the RFC to address and scope security. I agree this is important. I think this covers the bit you're mentioning about CI security; I think given the themes of the roadmap, TVM security should fall more into a "release-oriented" roadmap. Currently we haven't specified a roadmap to hold any work around release infra. We could expand this one to hold it, but I'd rather merge this so we can make forward progress on adding the CI & Testing tasks we have now to the existing roadmap, and contemplate a release roadmap in a follow-on RFC. I do indeed want to continue hacking on my poetry-based Python dependency management thing soon.

Could you clarify how security is limited to a release? The tooling we use to automate detection of insecure packages and vulnerable code should be ran across all changes rather than checking it as part of a release. We should aim to keep our own CI and development environments secure as a general practice with CI automation to aid us.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/54#issuecomment-1031287280
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>

Re: [apache/tvm-rfcs] [RFC][Roadmap] TVM Continuous Integration & Testing Roadmap (PR #54)

Posted by Christopher Sidebottom <no...@github.com.INVALID>.
@areusch helped me understand the contention here, there's a release process for managing things such as bugs, security vulnerabilities and other forms of fixes in a release cycle and accounting for all of those things in this roadmap is definitely out of scope :smile_cat: 

My confusion was that "Security problems with TVM dependencies" and "Security vulnerabilities in the TVM code itself" can be monitored by automation, which is what I'm suggesting to include as part of this roadmap. The actual tactics of remediation for issues would be defined elsewhere, likely alongside release processes or as part of a another roadmap.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/54#issuecomment-1031909240
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>

Re: [apache/tvm-rfcs] [RFC][Roadmap] TVM Continuous Integration & Testing Roadmap (PR #54)

Posted by Andrew Reusch <no...@github.com.INVALID>.
@leandron @Mousius thanks for taking a look! @denise-k updated the RFC to address and scope security. I agree this is important. I think this covers the bit you're mentioning about CI security; I think given the themes of the roadmap, TVM security should fall more into a "release-oriented" roadmap. Currently we haven't specified a roadmap to hold any work around release infra. We could expand this one to hold it, but I'd rather merge this so we can make forward progress on adding the CI & Testing tasks we have now to the existing roadmap, and contemplate a release roadmap in a follow-on RFC. I do indeed want to continue hacking on my poetry-based Python dependency management thing soon.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/54#issuecomment-1030441877
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>