You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/12/10 21:23:34 UTC
[GitHub] [tvm] jroesch opened a new pull request #7085: [WIP] Fixes for using Python APIs from Rust.
jroesch opened a new pull request #7085:
URL: https://github.com/apache/tvm/pull/7085
cc @hypercubestart this is a bigger PR to fix the current bindings and enable calling them with the Python code, but it needs a bit of cleanup, there are some changes that I used to debug segfaults.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] hypercubestart commented on a change in pull request #7085: Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
hypercubestart commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r578904690
##########
File path: rust/tvm/src/python.rs
##########
@@ -29,19 +29,30 @@ use pyo3::prelude::*;
pub fn load() -> Result<String, ()> {
let gil = Python::acquire_gil();
let py = gil.python();
+ // let main_mod = initialize();
+ //let main_mod = main_mod.as_ref(py);
load_python_tvm_(py).map_err(|e| {
// We can't display Python exceptions via std::fmt::Display,
// so print the error here manually.
e.print_and_set_sys_last_vars(py);
})
}
-// const TVMC_CODE: &'static str = include_str!("tvmc.py");
+pub fn import(mod_to_import: &str) -> PyResult<()> {
+ let gil = Python::acquire_gil();
+ let py = gil.python();
+ import_python(py, mod_to_import)?;
+ Ok(())
+}
+
+fn import_python<'p, 'b: 'p>(py: Python<'p>, to_import: &'b str) -> PyResult<&'p PyModule> {
+ let imported_mod = py.import(to_import)?;
+ Ok(imported_mod)
+}
fn load_python_tvm_(py: Python) -> PyResult<String> {
- let sys = py.import("tvm")?;
- let version: String = sys.get("__version__")?.extract()?;
- // py.run(TVMC_CODE, None, None)?;
+ let imported_mod = import_python(py, "tvm")?;
Review comment:
oops forgot to add the comment.
theres an ignored test `test_run` in this file, if everything works the ignore attribute should be removed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jroesch commented on a change in pull request #7085: [WIP] Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
jroesch commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r578132162
##########
File path: python/tvm/relay/build_module.py
##########
@@ -193,6 +193,18 @@ def get_params(self):
ret[key] = value.data
return ret
+@register_func("tvm.relay.build")
+def _rust_build_module(mod, target=None, target_host=None, params=None, mod_name="default"):
+ print(mod)
+ print("\n")
+ rt_mod = build(mod, target, target_host, params, mod_name).module
+ print(rt_mod)
+ print(rt_mod["default"])
Review comment:
leftover, going to try and land this now finally
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MarisaKirisame commented on a change in pull request #7085: Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
MarisaKirisame commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r580990981
##########
File path: rust/tvm/src/ir/expr.rs
##########
@@ -32,12 +32,14 @@ use super::span::Span;
#[type_key = "Expr"]
Review comment:
would it make sense to make a macro for the boilerplate?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jroesch commented on a change in pull request #7085: [WIP] Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
jroesch commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r578132071
##########
File path: rust/tvm/README.md
##########
@@ -15,221 +15,40 @@
<!--- specific language governing permissions and limitations -->
<!--- under the License. -->
-# TVM Runtime Frontend Support
+# TVM
-This crate provides an idiomatic Rust API for [TVM](https://github.com/apache/tvm) runtime frontend. Currently this requires **Nightly Rust** and tested on `rustc 1.32.0-nightly`
+This crate provides an idiomatic Rust API for [TVM](https://github.com/apache/tvm).
+The code works on **Stable Rust** and is tested against `rustc 1.47`.
-## What Does This Crate Offer?
-
-Here is a major workflow
-
-1. Train your **Deep Learning** model using any major framework such as [PyTorch](https://pytorch.org/), [Apache MXNet](https://mxnet.apache.org/) or [TensorFlow](https://www.tensorflow.org/)
-2. Use **TVM** to build optimized model artifacts on a supported context such as CPU, GPU, OpenCL and specialized accelerators.
-3. Deploy your models using **Rust** :heart:
-
-### Example: Deploy Image Classification from Pretrained Resnet18 on ImageNet1k
-
-Please checkout [examples/resnet](examples/resnet) for the complete end-to-end example.
-
-Here's a Python snippet for downloading and building a pretrained Resnet18 via Apache MXNet and TVM
-
-```python
-block = get_model('resnet18_v1', pretrained=True)
-
-sym, params = relay.frontend.from_mxnet(block, shape_dict)
-# compile the model
-with relay.build_config(opt_level=opt_level):
- graph, lib, params = relay.build(
- net, target, params=params)
-# same the model artifacts
-lib.save(os.path.join(target_dir, "deploy_lib.o"))
-cc.create_shared(os.path.join(target_dir, "deploy_lib.so"),
- [os.path.join(target_dir, "deploy_lib.o")])
-
-with open(os.path.join(target_dir, "deploy_graph.json"), "w") as fo:
- fo.write(graph.json())
-with open(os.path.join(target_dir,"deploy_param.params"), "wb") as fo:
- fo.write(relay.save_param_dict(params))
-```
+You can find the API Documentation [here](https://tvm.apache.org/docs/api/rust/tvm/index.html).
-Now, we need to input the artifacts to create and run the *Graph Runtime* to detect our input cat image
-
-
+## What Does This Crate Offer?
-as demostrated in the following Rust snippet
+The goal of this crate is to provide bindings to both the TVM compiler and runtime
+APIs. First train your **Deep Learning** model using any major framework such as
+[PyTorch](https://pytorch.org/), [Apache MXNet](https://mxnet.apache.org/) or [TensorFlow](https://www.tensorflow.org/).
+Then use **TVM** to build and deploy optimized model artifacts on a supported devices such as CPU, GPU, OpenCL and specialized accelerators.
-```rust
- let graph = fs::read_to_string("deploy_graph.json")?;
- // load the built module
- let lib = Module::load(&Path::new("deploy_lib.so"))?;
- // get the global TVM graph runtime function
- let runtime_create_fn = Function::get("tvm.graph_runtime.create", true).unwrap();
- let runtime_create_fn_ret = call_packed!(
- runtime_create_fn,
- &graph,
- &lib,
- &ctx.device_type,
- &ctx.device_id
- )?;
- // get graph runtime module
- let graph_runtime_module: Module = runtime_create_fn_ret.try_into()?;
- // get the registered `load_params` from runtime module
- let ref load_param_fn = graph_runtime_module
- .get_function("load_params", false)
- .unwrap();
- // parse parameters and convert to TVMByteArray
- let params: Vec<u8> = fs::read("deploy_param.params")?;
- let barr = TVMByteArray::from(¶ms);
- // load the parameters
- call_packed!(load_param_fn, &barr)?;
- // get the set_input function
- let ref set_input_fn = graph_runtime_module
- .get_function("set_input", false)
- .unwrap();
+The Rust bindings are composed of a few crates:
+- The [tvm](https://tvm.apache.org/docs/api/rust/tvm/index.html) crate which exposes Rust bindings to
+ both the compiler and runtime.
+- The [tvm_macros](https://tvm.apache.org/docs/api/rust/tvm/index.html) crate which provides macros
+ which generate unsafe boilerplate for TVM's data structures.
+- The [tvm_rt](https://tvm.apache.org/docs/api/rust/tvm_rt/index.html) crate which exposes Rust
Review comment:
TVM generates two dynamic libraries when we build it, the runtime bindings can be used to link to just the TVM runtime bindings, while the full tvm package links both the runtime and the compiler.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jroesch commented on a change in pull request #7085: Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
jroesch commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r581410349
##########
File path: rust/tvm/src/ir/expr.rs
##########
@@ -32,12 +32,14 @@ use super::span::Span;
#[type_key = "Expr"]
Review comment:
This is a macro 🤯
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] imalsogreg commented on a change in pull request #7085: [WIP] Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
imalsogreg commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r570241645
##########
File path: python/tvm/relay/analysis/__init__.py
##########
@@ -26,7 +26,7 @@
from . import call_graph
from .call_graph import CallGraph
-# Feature
+# # Feature
Review comment:
Two `#`'s?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MarisaKirisame commented on a change in pull request #7085: Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
MarisaKirisame commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r581415008
##########
File path: rust/tvm/src/ir/expr.rs
##########
@@ -32,12 +32,14 @@ use super::span::Span;
#[type_key = "Expr"]
Review comment:
I see. I just see the span boilerplate and thought you can refactor them. cool in this case then.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] imalsogreg commented on a change in pull request #7085: [WIP] Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
imalsogreg commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r570305029
##########
File path: rust/tvm/README.md
##########
@@ -15,221 +15,40 @@
<!--- specific language governing permissions and limitations -->
<!--- under the License. -->
-# TVM Runtime Frontend Support
+# TVM
-This crate provides an idiomatic Rust API for [TVM](https://github.com/apache/tvm) runtime frontend. Currently this requires **Nightly Rust** and tested on `rustc 1.32.0-nightly`
+This crate provides an idiomatic Rust API for [TVM](https://github.com/apache/tvm).
+The code works on **Stable Rust** and is tested against `rustc 1.47`.
-## What Does This Crate Offer?
-
-Here is a major workflow
-
-1. Train your **Deep Learning** model using any major framework such as [PyTorch](https://pytorch.org/), [Apache MXNet](https://mxnet.apache.org/) or [TensorFlow](https://www.tensorflow.org/)
-2. Use **TVM** to build optimized model artifacts on a supported context such as CPU, GPU, OpenCL and specialized accelerators.
-3. Deploy your models using **Rust** :heart:
-
-### Example: Deploy Image Classification from Pretrained Resnet18 on ImageNet1k
-
-Please checkout [examples/resnet](examples/resnet) for the complete end-to-end example.
-
-Here's a Python snippet for downloading and building a pretrained Resnet18 via Apache MXNet and TVM
-
-```python
-block = get_model('resnet18_v1', pretrained=True)
-
-sym, params = relay.frontend.from_mxnet(block, shape_dict)
-# compile the model
-with relay.build_config(opt_level=opt_level):
- graph, lib, params = relay.build(
- net, target, params=params)
-# same the model artifacts
-lib.save(os.path.join(target_dir, "deploy_lib.o"))
-cc.create_shared(os.path.join(target_dir, "deploy_lib.so"),
- [os.path.join(target_dir, "deploy_lib.o")])
-
-with open(os.path.join(target_dir, "deploy_graph.json"), "w") as fo:
- fo.write(graph.json())
-with open(os.path.join(target_dir,"deploy_param.params"), "wb") as fo:
- fo.write(relay.save_param_dict(params))
-```
+You can find the API Documentation [here](https://tvm.apache.org/docs/api/rust/tvm/index.html).
-Now, we need to input the artifacts to create and run the *Graph Runtime* to detect our input cat image
-
-
+## What Does This Crate Offer?
-as demostrated in the following Rust snippet
+The goal of this crate is to provide bindings to both the TVM compiler and runtime
+APIs. First train your **Deep Learning** model using any major framework such as
+[PyTorch](https://pytorch.org/), [Apache MXNet](https://mxnet.apache.org/) or [TensorFlow](https://www.tensorflow.org/).
+Then use **TVM** to build and deploy optimized model artifacts on a supported devices such as CPU, GPU, OpenCL and specialized accelerators.
-```rust
- let graph = fs::read_to_string("deploy_graph.json")?;
- // load the built module
- let lib = Module::load(&Path::new("deploy_lib.so"))?;
- // get the global TVM graph runtime function
- let runtime_create_fn = Function::get("tvm.graph_runtime.create", true).unwrap();
- let runtime_create_fn_ret = call_packed!(
- runtime_create_fn,
- &graph,
- &lib,
- &ctx.device_type,
- &ctx.device_id
- )?;
- // get graph runtime module
- let graph_runtime_module: Module = runtime_create_fn_ret.try_into()?;
- // get the registered `load_params` from runtime module
- let ref load_param_fn = graph_runtime_module
- .get_function("load_params", false)
- .unwrap();
- // parse parameters and convert to TVMByteArray
- let params: Vec<u8> = fs::read("deploy_param.params")?;
- let barr = TVMByteArray::from(¶ms);
- // load the parameters
- call_packed!(load_param_fn, &barr)?;
- // get the set_input function
- let ref set_input_fn = graph_runtime_module
- .get_function("set_input", false)
- .unwrap();
+The Rust bindings are composed of a few crates:
+- The [tvm](https://tvm.apache.org/docs/api/rust/tvm/index.html) crate which exposes Rust bindings to
+ both the compiler and runtime.
+- The [tvm_macros](https://tvm.apache.org/docs/api/rust/tvm/index.html) crate which provides macros
+ which generate unsafe boilerplate for TVM's data structures.
+- The [tvm_rt](https://tvm.apache.org/docs/api/rust/tvm_rt/index.html) crate which exposes Rust
Review comment:
Just wondering as a naive reader, why there is a `tvm_rt` crate when `tvm` exposes bindings to the runtime. (Is there _extra_ runtime stuff in `tmv_rt`, vs. `tvm_rt` being a subset of `tvm` for users that don't want to pull in the compiler bindings?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vegaluisjose commented on pull request #7085: Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
vegaluisjose commented on pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#issuecomment-789864343
There seems to be a problem with scipy in the CI @jroesch ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vegaluisjose merged pull request #7085: Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
vegaluisjose merged pull request #7085:
URL: https://github.com/apache/tvm/pull/7085
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] imalsogreg commented on a change in pull request #7085: [WIP] Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
imalsogreg commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r570251385
##########
File path: python/tvm/relay/build_module.py
##########
@@ -193,6 +193,18 @@ def get_params(self):
ret[key] = value.data
return ret
+@register_func("tvm.relay.build")
+def _rust_build_module(mod, target=None, target_host=None, params=None, mod_name="default"):
+ print(mod)
+ print("\n")
+ rt_mod = build(mod, target, target_host, params, mod_name).module
+ print(rt_mod)
+ print(rt_mod["default"])
Review comment:
leftover prints, or something we want to keep?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jroesch commented on a change in pull request #7085: Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
jroesch commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r580774293
##########
File path: rust/tvm/src/python.rs
##########
@@ -29,19 +29,30 @@ use pyo3::prelude::*;
pub fn load() -> Result<String, ()> {
let gil = Python::acquire_gil();
let py = gil.python();
+ // let main_mod = initialize();
+ //let main_mod = main_mod.as_ref(py);
load_python_tvm_(py).map_err(|e| {
// We can't display Python exceptions via std::fmt::Display,
// so print the error here manually.
e.print_and_set_sys_last_vars(py);
})
}
-// const TVMC_CODE: &'static str = include_str!("tvmc.py");
+pub fn import(mod_to_import: &str) -> PyResult<()> {
+ let gil = Python::acquire_gil();
+ let py = gil.python();
+ import_python(py, mod_to_import)?;
+ Ok(())
+}
+
+fn import_python<'p, 'b: 'p>(py: Python<'p>, to_import: &'b str) -> PyResult<&'p PyModule> {
+ let imported_mod = py.import(to_import)?;
+ Ok(imported_mod)
+}
fn load_python_tvm_(py: Python) -> PyResult<String> {
- let sys = py.import("tvm")?;
- let version: String = sys.get("__version__")?.extract()?;
- // py.run(TVMC_CODE, None, None)?;
+ let imported_mod = import_python(py, "tvm")?;
Review comment:
This test seems to pass on most platforms but not CI, going to keep disabled until I can be sure it won't cause CI flakiness.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] hypercubestart commented on a change in pull request #7085: Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
hypercubestart commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r587698328
##########
File path: rust/tvm/src/compiler/graph_rt.rs
##########
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+use std::convert::TryInto;
+use std::io::Read;
+use std::path::Path;
+
+use once_cell::sync::Lazy;
+use thiserror::Error;
+
+use crate::ir::IRModule;
+use crate::python;
+use crate::runtime::{map::Map, Function, Module as RtModule, NDArray, String};
+
+#[derive(Error, Debug)]
+pub enum Error {
+ #[error("{0}")]
+ IO(#[from] std::io::Error),
+ #[error("{0}")]
+ TVM(#[from] crate::errors::Error),
+}
+
+static TVM_BUILD: Lazy<Function> = Lazy::new(|| {
+ python::import("tvm").unwrap();
+ python::import("tvm.relay").unwrap();
+ Function::get("tvm.relay.build").unwrap()
+});
+
+fn _compile_module(
+ module: IRModule,
+ target: String,
+ target_host: String,
+ params: Map<String, NDArray>,
+ module_name: String,
+) -> Result<RtModule, Error> {
+ // The RAW API is Fn(IRModule, String, String, Map<String, NDArray>, String);
+ let module = TVM_BUILD.invoke(vec![
+ module.into(),
+ target.into(),
+ target_host.into(),
+ params.into(),
+ module_name.into(),
+ ])?;
+ let module: RtModule = module.try_into().unwrap();
+ Ok(module)
+}
+
+#[derive(Debug)]
+pub struct CompilerConfig {
+ target: Option<String>,
+ target_host: Option<String>,
+ params: Map<String, NDArray>,
+ module_name: Option<String>,
+}
+
+impl Default for CompilerConfig {
+ fn default() -> Self {
+ CompilerConfig {
+ target: None,
+ target_host: None,
+ params: Map::empty(),
+ module_name: None,
+ }
+ }
+}
+
+/// Compile a module from a configuration and IRModule.
+///
+/// # Arguments
+///
+/// * `config` - The configuration for the compiler.
+/// * `module` - The IRModule to compile.
+pub fn compile_module(config: CompilerConfig, module: IRModule) -> Result<RtModule, Error> {
Review comment:
is there a test for this?
##########
File path: rust/tvm/src/python.rs
##########
@@ -29,19 +29,30 @@ use pyo3::prelude::*;
pub fn load() -> Result<String, ()> {
let gil = Python::acquire_gil();
let py = gil.python();
+ // let main_mod = initialize();
+ //let main_mod = main_mod.as_ref(py);
load_python_tvm_(py).map_err(|e| {
// We can't display Python exceptions via std::fmt::Display,
// so print the error here manually.
e.print_and_set_sys_last_vars(py);
})
}
-// const TVMC_CODE: &'static str = include_str!("tvmc.py");
+pub fn import(mod_to_import: &str) -> PyResult<()> {
+ let gil = Python::acquire_gil();
+ let py = gil.python();
+ import_python(py, mod_to_import)?;
+ Ok(())
+}
+
+fn import_python<'p, 'b: 'p>(py: Python<'p>, to_import: &'b str) -> PyResult<&'p PyModule> {
+ let imported_mod = py.import(to_import)?;
+ Ok(imported_mod)
+}
fn load_python_tvm_(py: Python) -> PyResult<String> {
- let sys = py.import("tvm")?;
- let version: String = sys.get("__version__")?.extract()?;
- // py.run(TVMC_CODE, None, None)?;
+ let imported_mod = import_python(py, "tvm")?;
Review comment:
ok, its strange to me that loading tvm into rust is flaky though
##########
File path: rust/tvm-rt/src/module.rs
##########
@@ -26,21 +26,24 @@ use std::{
ptr,
};
+use crate::object::Object;
+use tvm_macros::Object;
use tvm_sys::ffi;
use crate::errors::Error;
+use crate::String as TString;
use crate::{errors, function::Function};
-const ENTRY_FUNC: &str = "__tvm_main__";
-
/// Wrapper around TVM module handle which contains an entry function.
/// The entry function can be applied to an imported module through [`entry_func`].
///
/// [`entry_func`]:struct.Module.html#method.entry_func
Review comment:
is this comment still true?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jroesch commented on pull request #7085: [WIP] Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
jroesch commented on pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#issuecomment-773037810
cc @hypercubestart @binarybana @imalsogreg updated this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] imalsogreg commented on a change in pull request #7085: [WIP] Fixes for using Python APIs from Rust.
Posted by GitBox <gi...@apache.org>.
imalsogreg commented on a change in pull request #7085:
URL: https://github.com/apache/tvm/pull/7085#discussion_r570241645
##########
File path: python/tvm/relay/analysis/__init__.py
##########
@@ -26,7 +26,7 @@
from . import call_graph
from .call_graph import CallGraph
-# Feature
+# # Feature
Review comment:
Two `#`'s?
##########
File path: python/tvm/relay/build_module.py
##########
@@ -193,6 +193,18 @@ def get_params(self):
ret[key] = value.data
return ret
+@register_func("tvm.relay.build")
+def _rust_build_module(mod, target=None, target_host=None, params=None, mod_name="default"):
+ print(mod)
+ print("\n")
+ rt_mod = build(mod, target, target_host, params, mod_name).module
+ print(rt_mod)
+ print(rt_mod["default"])
Review comment:
leftover prints, or something we want to keep?
##########
File path: rust/tvm/README.md
##########
@@ -15,221 +15,40 @@
<!--- specific language governing permissions and limitations -->
<!--- under the License. -->
-# TVM Runtime Frontend Support
+# TVM
-This crate provides an idiomatic Rust API for [TVM](https://github.com/apache/tvm) runtime frontend. Currently this requires **Nightly Rust** and tested on `rustc 1.32.0-nightly`
+This crate provides an idiomatic Rust API for [TVM](https://github.com/apache/tvm).
+The code works on **Stable Rust** and is tested against `rustc 1.47`.
-## What Does This Crate Offer?
-
-Here is a major workflow
-
-1. Train your **Deep Learning** model using any major framework such as [PyTorch](https://pytorch.org/), [Apache MXNet](https://mxnet.apache.org/) or [TensorFlow](https://www.tensorflow.org/)
-2. Use **TVM** to build optimized model artifacts on a supported context such as CPU, GPU, OpenCL and specialized accelerators.
-3. Deploy your models using **Rust** :heart:
-
-### Example: Deploy Image Classification from Pretrained Resnet18 on ImageNet1k
-
-Please checkout [examples/resnet](examples/resnet) for the complete end-to-end example.
-
-Here's a Python snippet for downloading and building a pretrained Resnet18 via Apache MXNet and TVM
-
-```python
-block = get_model('resnet18_v1', pretrained=True)
-
-sym, params = relay.frontend.from_mxnet(block, shape_dict)
-# compile the model
-with relay.build_config(opt_level=opt_level):
- graph, lib, params = relay.build(
- net, target, params=params)
-# same the model artifacts
-lib.save(os.path.join(target_dir, "deploy_lib.o"))
-cc.create_shared(os.path.join(target_dir, "deploy_lib.so"),
- [os.path.join(target_dir, "deploy_lib.o")])
-
-with open(os.path.join(target_dir, "deploy_graph.json"), "w") as fo:
- fo.write(graph.json())
-with open(os.path.join(target_dir,"deploy_param.params"), "wb") as fo:
- fo.write(relay.save_param_dict(params))
-```
+You can find the API Documentation [here](https://tvm.apache.org/docs/api/rust/tvm/index.html).
-Now, we need to input the artifacts to create and run the *Graph Runtime* to detect our input cat image
-
-
+## What Does This Crate Offer?
-as demostrated in the following Rust snippet
+The goal of this crate is to provide bindings to both the TVM compiler and runtime
+APIs. First train your **Deep Learning** model using any major framework such as
+[PyTorch](https://pytorch.org/), [Apache MXNet](https://mxnet.apache.org/) or [TensorFlow](https://www.tensorflow.org/).
+Then use **TVM** to build and deploy optimized model artifacts on a supported devices such as CPU, GPU, OpenCL and specialized accelerators.
-```rust
- let graph = fs::read_to_string("deploy_graph.json")?;
- // load the built module
- let lib = Module::load(&Path::new("deploy_lib.so"))?;
- // get the global TVM graph runtime function
- let runtime_create_fn = Function::get("tvm.graph_runtime.create", true).unwrap();
- let runtime_create_fn_ret = call_packed!(
- runtime_create_fn,
- &graph,
- &lib,
- &ctx.device_type,
- &ctx.device_id
- )?;
- // get graph runtime module
- let graph_runtime_module: Module = runtime_create_fn_ret.try_into()?;
- // get the registered `load_params` from runtime module
- let ref load_param_fn = graph_runtime_module
- .get_function("load_params", false)
- .unwrap();
- // parse parameters and convert to TVMByteArray
- let params: Vec<u8> = fs::read("deploy_param.params")?;
- let barr = TVMByteArray::from(¶ms);
- // load the parameters
- call_packed!(load_param_fn, &barr)?;
- // get the set_input function
- let ref set_input_fn = graph_runtime_module
- .get_function("set_input", false)
- .unwrap();
+The Rust bindings are composed of a few crates:
+- The [tvm](https://tvm.apache.org/docs/api/rust/tvm/index.html) crate which exposes Rust bindings to
+ both the compiler and runtime.
+- The [tvm_macros](https://tvm.apache.org/docs/api/rust/tvm/index.html) crate which provides macros
+ which generate unsafe boilerplate for TVM's data structures.
+- The [tvm_rt](https://tvm.apache.org/docs/api/rust/tvm_rt/index.html) crate which exposes Rust
Review comment:
Just wondering as a naive reader, why there is a `tvm_rt` crate when `tvm` exposes bindings to the runtime. (Is there _extra_ runtime stuff in `tmv_rt`, vs. `tvm_rt` being a subset of `tvm` for users that don't want to pull in the compiler bindings?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org