You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/07/06 16:15:51 UTC
[GitHub] [incubator-tvm] tqchen commented on a change in pull request #5753: [Draft] Support Module based interface runtime
tqchen commented on a change in pull request #5753:
URL: https://github.com/apache/incubator-tvm/pull/5753#discussion_r450315280
##########
File path: python/tvm/contrib/graph_runtime.py
##########
@@ -63,6 +63,15 @@ def create(graph_json_str, libmod, ctx):
return GraphModule(fcreate(graph_json_str, libmod, *device_type_id))
+# TODO (FrozenGene): rename
+def create4unified(libmod, ctx):
+ ctx, num_rpc_ctx, device_type_id = get_device_ctx(libmod, ctx)
Review comment:
This function is not needed, given that it is simple enough to directly call the constructor function. Let us think about another way of wrapping.
For example: the following code should work for both RPC and local
```python
gmod = graph_runtime.GraphModule(complied_graph_lib['resnet18'](cpu_ctx))
gmod.set_input()
```
##########
File path: python/tvm/rpc/client.py
##########
@@ -160,7 +160,12 @@ def load_module(self, path):
m : Module
The remote module containing remote function.
"""
- return _ffi_api.LoadRemoteModule(self._sess, path)
+ module = _ffi_api.LoadRemoteModule(self._sess, path)
Review comment:
Let us not do special handling here, as GraphRuntimeFactoryModule itself is only needed for backward compatibility reasons.
The return value of RPC should always be simple -- a runtime.Module and we can do wrapping on the outside after we get the Module, instead of doing automatic wrapping in here.
##########
File path: python/tvm/runtime/module.py
##########
@@ -222,29 +224,31 @@ def evaluator(*args):
except NameError:
raise NameError("time_evaluate is only supported when RPC is enabled")
- def _collect_dso_modules(self):
- """Helper function to collect dso modules, then return it."""
- visited, stack, dso_modules = set(), [], []
+ def _collect_modules(self, module_type_keys):
+ """Helper function to collect specific modules, then return it."""
+ visited, stack, modules = set(), [], []
+ type_keys = module_type_keys if isinstance(module_type_keys, (list, tuple)) else [module_type_keys]
# append root module
visited.add(self)
stack.append(self)
while stack:
module = stack.pop()
- if module._dso_exportable():
- dso_modules.append(module)
+ if module.type_key in type_keys:
+ modules.append(module)
for m in module.imported_modules:
if m not in visited:
visited.add(m)
stack.append(m)
- return dso_modules
+ return modules
- def _dso_exportable(self):
- return self.type_key == "llvm" or self.type_key == "c"
+ def _dso_exportable_types(self):
+ return ["llvm", "c"]
Review comment:
use tuple
##########
File path: python/tvm/runtime/graph_runtime_factory.py
##########
@@ -0,0 +1,144 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Graph runtime factory."""
+import numpy as np
+import warnings
+from tvm._ffi.base import string_types
+from tvm._ffi.registry import get_global_func
+from tvm._ffi.runtime_ctypes import TVMContext
+from tvm.contrib.graph_runtime import get_device_ctx
+from .packed_func import _set_class_module
+from tvm.rpc import base as rpc_base
+from .module import Module
+from . import ndarray
+
+
+def create(graph_runtime_kind, graph_json_str, libmod, params, module_name='default'):
+ """Create a runtime executor module given a graph and module.
+ Parameters
+ ----------
+ graph_runtime_kind: str
+ The kind of graph runtime. Like graphruntime, vm and so on.
+ graph_json_str : str or graph class
+ The graph to be deployed in json format output by nnvm graph.
+ The graph can only contain one operator(tvm_op) that
+ points to the name of PackedFunc in the libmod.
+ libmod : tvm.Module
+ The module of the corresponding function
+ Returns
+ -------
+ graph_module : GraphModule
+ Runtime graph module that can be used to execute the graph.
+ """
+ if not isinstance(graph_json_str, string_types):
+ try:
+ graph_json_str = graph_json_str._tvm_graph_json()
+ except AttributeError:
+ raise ValueError("Type %s is not supported" % type(graph_json_str))
+ fcreate = get_global_func("tvm.graph_runtime_factory.create")
+ args = []
+ for k, v in params.items():
+ args.append(k)
+ args.append(ndarray.array(v))
+ return GraphRuntimeFactoryModule(fcreate(graph_runtime_kind, graph_json_str, libmod, module_name, *args))
+
+
+class GraphRuntimeFactoryModule(Module):
+ """Graph runtime factory module.
+
+ This is a module of graph runtime factory
+
+ Parameters
+ ----------
+ module : Module
+ The interal tvm module that holds the actual graph functions.
+
+ Attributes
+ ----------
+ module : Module
+ The interal tvm module that holds the actual graph functions.
+ """
+
+ def __init__(self, module):
+ self.module = module
+ self._select_module = module["select_module"]
+ self._import_module = module["import_module"]
+ self.selected_module = None
+ self.graph_json = None
+ self.lib = None
+ self.params = {}
+ self.iter_cnt = 0
+ super(GraphRuntimeFactoryModule, self).__init__(self.module.handle)
+
+ def __del__(self):
Review comment:
no need to handle Module.
##########
File path: src/runtime/graph/graph_runtime_factory.h
##########
@@ -0,0 +1,133 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/runtime/graph_runtime_factory.h
+ * \brief Graph runtime factory creating graph runtime.
+ */
+
+#ifndef TVM_RUNTIME_GRAPH_GRAPH_RUNTIME_FACTORY_H_
+#define TVM_RUNTIME_GRAPH_GRAPH_RUNTIME_FACTORY_H_
+
+#include <tvm/runtime/c_runtime_api.h>
+#include <tvm/runtime/module.h>
+#include <tvm/runtime/ndarray.h>
+#include <tvm/runtime/packed_func.h>
+
+#include <string>
+#include <unordered_map>
+#include <vector>
+
+namespace tvm {
+namespace runtime {
+
+class TVM_DLL GraphRuntimeFactory : public runtime::ModuleNode {
+ public:
+ /*!
+ * \brief Initialize the GraphRuntimeFactory with graph and context.
+ * \param graph_json The execution graph.
+ * \param params The params of graph.
+ * \param kind The runtime kind to be created.
+ */
+ void Init(const std::string& kind, const std::string& graph_json,
+ const std::unordered_map<std::string, tvm::runtime::NDArray>& params,
+ const std::string& module_name = "default");
+
+ /*!
+ * \brief Import other GraphRuntimeFactory module.
+ * \param other The GraphRuntimeFactory module we want to import.
+ */
+ void ImportModule(Module other);
+
+ /*!
+ * \brief Get member function to front-end
+ * \param name The name of the function.
+ * \param sptr_to_self The pointer to the module node.
+ * \return The corresponding member function.
+ */
+ virtual PackedFunc GetFunction(const std::string& name, const ObjectPtr<Object>& sptr_to_self);
Review comment:
final instead of virtual, as we want to mark the final override.
##########
File path: python/tvm/runtime/graph_runtime_factory.py
##########
@@ -0,0 +1,144 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Graph runtime factory."""
+import numpy as np
+import warnings
+from tvm._ffi.base import string_types
+from tvm._ffi.registry import get_global_func
+from tvm._ffi.runtime_ctypes import TVMContext
+from tvm.contrib.graph_runtime import get_device_ctx
+from .packed_func import _set_class_module
+from tvm.rpc import base as rpc_base
+from .module import Module
+from . import ndarray
+
+
+def create(graph_runtime_kind, graph_json_str, libmod, params, module_name='default'):
+ """Create a runtime executor module given a graph and module.
+ Parameters
+ ----------
+ graph_runtime_kind: str
+ The kind of graph runtime. Like graphruntime, vm and so on.
+ graph_json_str : str or graph class
+ The graph to be deployed in json format output by nnvm graph.
+ The graph can only contain one operator(tvm_op) that
+ points to the name of PackedFunc in the libmod.
+ libmod : tvm.Module
+ The module of the corresponding function
+ Returns
+ -------
+ graph_module : GraphModule
+ Runtime graph module that can be used to execute the graph.
+ """
+ if not isinstance(graph_json_str, string_types):
+ try:
+ graph_json_str = graph_json_str._tvm_graph_json()
+ except AttributeError:
+ raise ValueError("Type %s is not supported" % type(graph_json_str))
+ fcreate = get_global_func("tvm.graph_runtime_factory.create")
+ args = []
+ for k, v in params.items():
+ args.append(k)
+ args.append(ndarray.array(v))
+ return GraphRuntimeFactoryModule(fcreate(graph_runtime_kind, graph_json_str, libmod, module_name, *args))
+
+
+class GraphRuntimeFactoryModule(Module):
+ """Graph runtime factory module.
+
+ This is a module of graph runtime factory
+
+ Parameters
+ ----------
+ module : Module
+ The interal tvm module that holds the actual graph functions.
+
+ Attributes
+ ----------
+ module : Module
+ The interal tvm module that holds the actual graph functions.
+ """
+
+ def __init__(self, module):
+ self.module = module
+ self._select_module = module["select_module"]
+ self._import_module = module["import_module"]
+ self.selected_module = None
+ self.graph_json = None
+ self.lib = None
+ self.params = {}
+ self.iter_cnt = 0
+ super(GraphRuntimeFactoryModule, self).__init__(self.module.handle)
+
+ def __del__(self):
+ pass
+
+ def runtime_create(self, ctx):
Review comment:
Remove ths function as we can directly do `mod["default"](ctx)`
##########
File path: src/runtime/graph/graph_runtime_factory.h
##########
@@ -0,0 +1,133 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/runtime/graph_runtime_factory.h
+ * \brief Graph runtime factory creating graph runtime.
+ */
+
+#ifndef TVM_RUNTIME_GRAPH_GRAPH_RUNTIME_FACTORY_H_
+#define TVM_RUNTIME_GRAPH_GRAPH_RUNTIME_FACTORY_H_
+
+#include <tvm/runtime/c_runtime_api.h>
+#include <tvm/runtime/module.h>
+#include <tvm/runtime/ndarray.h>
+#include <tvm/runtime/packed_func.h>
+
+#include <string>
+#include <unordered_map>
+#include <vector>
+
+namespace tvm {
+namespace runtime {
+
+class TVM_DLL GraphRuntimeFactory : public runtime::ModuleNode {
+ public:
+ /*!
+ * \brief Initialize the GraphRuntimeFactory with graph and context.
+ * \param graph_json The execution graph.
+ * \param params The params of graph.
+ * \param kind The runtime kind to be created.
+ */
+ void Init(const std::string& kind, const std::string& graph_json,
+ const std::unordered_map<std::string, tvm::runtime::NDArray>& params,
+ const std::string& module_name = "default");
+
+ /*!
+ * \brief Import other GraphRuntimeFactory module.
+ * \param other The GraphRuntimeFactory module we want to import.
+ */
+ void ImportModule(Module other);
+
+ /*!
+ * \brief Get member function to front-end
+ * \param name The name of the function.
+ * \param sptr_to_self The pointer to the module node.
+ * \return The corresponding member function.
+ */
+ virtual PackedFunc GetFunction(const std::string& name, const ObjectPtr<Object>& sptr_to_self);
+
+ /*!
+ * \return The type key of the executor.
+ */
+ const char* type_key() const override { return "GraphRuntimeFactory"; }
+
+ /*!
+ * \brief Save the module to binary stream.
+ * \param stream The binary stream to save to.
+ */
+ void SaveToBinary(dmlc::Stream* stream) override;
+
+ /*!
+ * \brief Create a specific runtime module
+ * \param module The module we will be used for creating runtime
+ * \param ctxs The context of the host and devices where graph nodes will be
+ * executed on.
+ * \return created runtime module
+ */
+ Module RuntimeCreate(Module module, const std::vector<TVMContext>& ctxs);
+
+ /*!
+ * \brief Select the specific module
+ * \param name The name of the module
+ * \return selected module
+ */
+ Module SelectModule(const std::string& name);
+
+ const std::string& GetJson() const { return graph_json_; }
+
+ std::unordered_map<std::string, tvm::runtime::NDArray> GetParams() const { return params_; }
+
+ Module GetLib() const {
+ CHECK_GT(this->imports().size(), 0);
+ return this->imports_[0];
+ }
+
+ const std::string& GetKind() const { return kind_; }
Review comment:
I am not sure if we need the kind argument here, I understand the goal is for polymorphism, however, we might use a quite different way for packaging in VM and only expose the same set of functions(aka create)
##########
File path: src/runtime/graph/graph_runtime.h
##########
@@ -171,6 +175,36 @@ class TVM_DLL GraphRuntime : public ModuleNode {
std::string GetNodeName(uint32_t nid) const { return nodes_[nid].name; }
+ /*!
+ * \brief Set the graph params.
+ * \param params The graph params value we want to set.
+ */
+ void SetParams(const std::unordered_map<std::string, tvm::runtime::NDArray>& params) {
Review comment:
Move this function to GraphRuntimeFactory as we can build it in the outside, also avoids a reverse dep on the graph runtime factory.
##########
File path: python/tvm/runtime/module.py
##########
@@ -282,7 +293,22 @@ def export_library(self,
self.save(file_name)
return
- modules = self._collect_dso_modules()
+ graph_runtime_factory_modules = self._collect_modules("GraphRuntimeFactory")
+ for index, module in enumerate(graph_runtime_factory_modules):
+ if not package_params:
+ module.get_function("diable_package_params")()
+ params_file_name = "deploy_" + module.get_function("get_module_name")() + ".params"
Review comment:
We should not add any logics specific to GraphRuntimeFactory here, and directly use the original serialization mechanism. The additional behavior(save `params` if not packaging params) brings inconsistency to the API. We would rather let the user call `get_params` by themselves and save it in this case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org