[GitHub] [tvm] jwfromm commented on a change in pull request #10722: [TVMC] Support compiling and running with VM

[GitBox <gi...@apache.org>]

jwfromm commented on a change in pull request #10722:
URL: https://github.com/apache/tvm/pull/10722#discussion_r833623465



##########
File path: python/tvm/driver/tvmc/runner.py
##########
@@ -530,58 +531,82 @@ def run_module(
             assert device == "cpu"
             dev = session.cpu()
 
-        # TODO(gromero): Adjust for micro targets.
-        if profile:
-            logger.debug("Creating runtime with profiling enabled.")
-            module = debug_executor.create(tvmc_package.graph, lib, dev, dump_root="./prof")
+        if tvmc_package.use_vm:
+            assert inputs is not None and isinstance(
+                inputs, dict
+            ), "vm runner requires inputs to be provided as a dict"
+            exe = vm.VirtualMachine(lib, dev)
+            input_tensor = {}
+            for e, i in inputs.items():
+                input_tensor[e] = tvm.nd.array(i, dev)
+            exe.set_input("main", **input_tensor)
+            exe.invoke_stateful("main")
+            times = exe.benchmark(
+                dev,
+                **input_tensor,
+                func_name="main",
+                repeat=repeat,
+                number=number,
+                end_to_end=end_to_end,
+            )
+            exe_outputs = exe.get_outputs()
+            outputs = {}
+            for i, val in enumerate(exe_outputs):
+                output_name = "output_{}".format(i)
+                outputs[output_name] = val
         else:
-            if device == "micro":
-                logger.debug("Creating runtime (micro) with profiling disabled.")
-                module = tvm.micro.create_local_graph_executor(tvmc_package.graph, lib, dev)
+            # TODO(gromero): Adjust for micro targets.
+            if profile:

Review comment:
       The VM also supports profiling using `VirtualMachineProfiler`. We should support that in runner.

##########
File path: python/tvm/driver/tvmc/compiler.py
##########
@@ -243,7 +244,8 @@ def compile_model(
         PassContext.
     additional_target_options: Optional[Dict[str, Dict[str, Any]]]
         Additional target options in a dictionary to combine with initial Target arguments
-
+    use_vm: bool

Review comment:
       After reading this code a little more, I think it would make sense to drop the `use_vm` argument, and instead specify that you should compile and run with the VM by setting `executor=Executor("vm")` instead of `executor=Executor("graph")` by default. I think this is consistent with the intention of Executor.

##########
File path: python/tvm/driver/tvmc/runner.py
##########
@@ -530,58 +531,82 @@ def run_module(
             assert device == "cpu"
             dev = session.cpu()
 
-        # TODO(gromero): Adjust for micro targets.
-        if profile:
-            logger.debug("Creating runtime with profiling enabled.")
-            module = debug_executor.create(tvmc_package.graph, lib, dev, dump_root="./prof")
+        if tvmc_package.use_vm:

Review comment:
       Unfortunately the VM doesnt track any information about the inputs, presumably because its geared more towards dynamism. I think we'll just have to check that inputs are explicitly provided.

##########
File path: python/tvm/driver/tvmc/model.py
##########
@@ -337,7 +385,21 @@ def import_package(self, package_path: str):
         t = tarfile.open(package_path)
         t.extractall(temp.relpath("."))
 
-        if os.path.exists(temp.relpath("metadata.json")):
+        if self.use_vm:

Review comment:
       Instead of requiring users to remember if a saved package uses the VM or not, maybe we can combine the VM handling with the classic format handling and set `self.use_vm` if we find `lib.tar` and `lib.so` instead of `mod.tar` and `mod.so`. 

##########
File path: python/tvm/driver/tvmc/model.py
##########
@@ -248,11 +284,12 @@ def export_classic_format(
 
     def export_package(
         self,
-        executor_factory: GraphExecutorFactoryModule,
+        executor_factory: Union[GraphExecutorFactoryModule, tvm.runtime.vm.Executable],

Review comment:
       To look prettier we can add `from tvm.runtime.vm import Executable` up top then just use `Executable` here.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org