[dmlc/tvm] [RFC] MISRA-C 2012 Compliant C Runtime (#3159)

[Liangfu Chen <no...@github.com>]

For a long time, the development of TVM has been focused on optimizing deep learning inference for almost any processing systems, and it has been used widely in many applications fields. However, when users of TVM were trying to use theirs pretrained & quanitzed model in critical systems (e.g. aircraft control and autonomous driving), the running source code is expected to be extremely reliable and compact. This RFC intends to bring and discuss the possibility to port an essential part of the runtime code base to MISRA-C compliant form, so that the pretrained & quantized model can be potentially used in safety critical systems.

### Features Proposed to be MISRA-C Compliant
* `load_json` and `load_params` related functions
* `ndarray` class
* `graph_runtime` class
* Interface with Zephyr RTOS (e.g. memory allocation, storage device, network related)

### Proposed Changes
* MISRA-C compliant code should be available at `src/crt` directory
* Test cases should be created to check consistency between c++ implement and MISRA-C compliant versions.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159

Re: [dmlc/tvm] [RFC] MISRA-C/C++ Compliant Runtime (#3159)

[Nick Hynes <no...@github.com>]

> minimal Rust runtime which might be useful as well here?

For sure. The Rust runtime has all of the features required for running syslib modules, but it makes use of `alloc` and some tasteful unsafety, which are disallowed by MISRA. If we really wanted to have an ultra-safe runtime, we could do a bit better there. Rust is probably closer to complete safety than rewriting the C++ runtime, though.

Of course, as @Ravenwater has oft mentioned, adopting Rust for existing projects is something of a nonstarter. Perhaps if we could enumerate which platforms (and users) would want a Safe runtime, we could determine whether C++ is truly necessary or just the first thing that comes to mind.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159#issuecomment-492883850

Re: [dmlc/tvm] [RFC] MISRA-C/C++ Compliant Runtime (#3159)

[Nick Hynes <no...@github.com>]

> mimic device-alloc with the fix size heap

From [what](https://cboard.cprogramming.com/c-programming/124871-what-alternative-malloc.html) I [can](https://misra.org.uk/forum/viewtopic.php?t=260) [tell](https://stackoverflow.com/questions/16740435/c-memory-management-and-misra), writing a user `malloc` is against the spirit of MISRA. Part of the benefit is knowing _exactly_ what will happen in the memory manager, but I'm not sure whether it's reasonable to ask people to bring their own libc. We actually don't need dynamic memory management at all, though.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159#issuecomment-493266723

Re: [dmlc/tvm] [RFC] MISRA-C/C++ Compliant Runtime (#3159)

[Liangfu Chen <no...@github.com>]

> Does MISRA compliance affect the compiler or the generated code as well?

In my observation, the compliance does affect the compiler and the code generator. However, as a first step towards making TVM available for the critical and safety related systems, I think it's important to consider the runtime design first.

> Perhaps if we could enumerate which platforms (and users) would want a Safe runtime

It's difficult to enumerate all the platforms and users who want such runtime. However, it would be a good start to consider how could a developer use TVM to perform the perception task in a fully automated driving vehicle. There are a number of platforms that are already certified by ISO 26262 (e.g. [QNX](http://www.qnx.com/news/pr_5603_1.html) ), and for the open source world, I think it's good to start with Zephyr RTOS.

On the other hand, in order to reduce the effort in rewriting another minimal runtime, I think we should try to eliminate the usage of `malloc`, or at least conclude required memory allocation to input, intermediate and output buffers only.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159#issuecomment-492896292

Re: [dmlc/tvm] [RFC] MISRA-C/C++ Compliant Runtime (#3159)

[Tianqi Chen <no...@github.com>]

I don't mean to bring our own libc, but just to implement the DeviceAPI::Alloc with a fixed pre-allocated heap.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159#issuecomment-493274651

Re: [dmlc/tvm] [RFC] MISRA-C/C++ Compliant Runtime (#3159)

[Nick Hynes <no...@github.com>]

It's definitely possible to fixup the TVM runtime so that the buffers are included as part of the static library and to inline the pointers at compile time. If the graph is also constructed at compile time (something I've been wanting to do in the rust runtime anyway) then there's not much outside of `(Workspace|Thread)Pool` that needs allocation.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159#issuecomment-493258609

Re: [dmlc/tvm] [RFC] MISRA-C/C++ Compliant Runtime (#3159)

[Andrew Tulloch <no...@github.com>]

@liangfu this looks quite interesting. Does MISRA compliance affect the compiler or the generated code as well (i.e. how do we certify that the output of e.g. LLVM is correct, doesn't segfault, etc?).

WRT to the minimal runtime I put together that `bundle_deploy.cc` list quite quickly, I think with a bit of pruning (e.g. if we enforce single threaded use from MISRA, we can get rid of `threading_backend`/`thread_pool`, etc). IIRC @nhynes and co were potentially interested in a minimal Rust runtime which might be useful as well here?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159#issuecomment-492850849

Re: [dmlc/tvm] [RFC] MISRA-C/C++ Compliant Runtime (#3159)

[Tianqi Chen <no...@github.com>]

Memory allocation is fine, as long as we have a pre-allocated heap that provide a sufficient amount of meory

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159#issuecomment-492906155

Re: [dmlc/tvm] [RFC] MISRA-C/C++ Compliant Runtime (#3159)

[Liangfu Chen <no...@github.com>]

A software implemented with `malloc` would only reach functional safety level of QM. It would be far from to be graded as ASIL-D. See ISO 26262 Table 8 Topic 2 in [Meeting ISO 26262 Guidelines with the Synopsys Software Integrity Portfolio](https://www.synopsys.com/content/dam/synopsys/sig-assets/whitepapers/meeting-iso-26262-software-standards.pdf)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159#issuecomment-492955919

Re: [dmlc/tvm] [RFC] MISRA-C/C++ Compliant Runtime (#3159)

[Tianqi Chen <no...@github.com>]

Yes, what I mean is to pre-calculate the space necessary and mimic device-alloc with the fix size heap

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159#issuecomment-493259710

Re: [dmlc/tvm] [RFC] MISRA-C/C++ Compliant Runtime (#3159)

[Liangfu Chen <no...@github.com>]

If I understand correctly, using a fixed pre-allocated heap would lead us to convert a fragment of the storage pool (which is a fixed-sized static memory) into the specific data type for computation. This violates another rule that restrict (or avoid) using data type conversion. In this case, I think building static graph is preferred.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159#issuecomment-494735122

Re: [dmlc/tvm] [RFC] MISRA-C/C++ Compliant Runtime (#3159)

[Nick Hynes <no...@github.com>]

Ah, that makes total sense: a non-trivial implementation of DeviceAPI. Gee, what a nice abstraction! I still kind of want the static graph, though :)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3159#issuecomment-493287722