[Apache TVM Discuss] [Questions] How to set parameter into pipeline module

[popojames via Apache TVM Discuss <no...@discuss.tvm.ai>]

Hello @hijiang,

I have followed your work and made some extensions to support such splitting as I mentioned in the previous discussions https://discuss.tvm.apache.org/t/setting-the-cpu-affinity-and-number-of-cores-locally-without-rpc-remote/11306/4?u=popojames. https://discuss.tvm.apache.org/t/can-tvm-split-work-into-different-layers-and-assign-layers-into-different-cores/11161/10?u=popojames. I was able to split the network and with my own CPU affinity setting.

Now I try to assign **param** of the pre-trained model into the pipeline module,
I noticed the function of PipelineModule.set_input support such setting:

https://github.com/apache/tvm/blob/fba531e55f032367acc28090605c5533b2e88f55/python/tvm/contrib/pipeline_executor.py#L222

However, this function will induce the following error:

`AttributeError: 'Module' object has no attribute 'set_input'`

This is because of the type of **self.graph_modules_[mod_idx]** is something like **Module(GraphExecutor, 27517208)** instead of **tvm.contrib.graph_executor.GraphModule**. In this case, the latter can access set_input but the former cannot.

Is there any other way to assign parameters or can I transfer from Module(GraphExecutor, XXX) to tvm.contrib.graph_executor.GraphModule in this case to set the parameters?

Thanks for your help in advance and many thanks for developing this TVM function.





---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-set-parameter-into-pipeline-module/11375/1) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/72ba1c0886e2a25d30b8d9e20513905384583a7928c4c2b7cbe102f84022b569).

[Apache TVM Discuss] [Questions] How to set parameter into pipeline module

[popojames via Apache TVM Discuss <no...@discuss.tvm.ai>]

Hello @hjiang,

Thanks for answering, I was able to feed the params into the model by adopting your change.

But I add one more change: I changed to the following code
>     if params:
>          for param in params:
>              self.graph_modules_[mod_idx].set_input(**params)
 into the following code to avoid the error 
>     if params:
>          self.graph_modules_[mod_idx].set_input(**params)

https://github.com/apache/tvm/blob/1ee5831b61b5a196eb99a44a998087f3c9c8c4a1/python/tvm/contrib/pipeline_executor.py#L220

====================================================================

More info:
What I want to do is very similar to do what you have done in [tvm/test_pipeline_executor.py at fba531e55f032367acc28090605c5533b2e88f55 · apache/tvm (github.com)](https://github.com/apache/tvm/blob/fba531e55f032367acc28090605c5533b2e88f55/tests/python/relay/test_pipeline_executor.py)
To be more specific, I tried to compare the correctness of the pipeline setting, in another word, if I feed the input to with pipeline and run the inference, I was expected to get the same result as I get in without the pipeline setting. Now I was able to achieve so.

Thanks again :)





---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-set-parameter-into-pipeline-module/11375/4) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/e1a7b7b02629082440aa98c6c5c91ceaa8bf87615862c685390bb6a8a1c8ca9b).

[Apache TVM Discuss] [Questions] How to set parameter into pipeline module

[huajsj via Apache TVM Discuss <no...@discuss.tvm.ai>]

@popojames , thanks for trying the pipeline runtime, the set params issue will get address in our new patch, you also can try following change as a quick fix.
```
diff --git a/python/tvm/contrib/pipeline_executor.py b/python/tvm/contrib/pipeline_executor.py
index bd8c9a768..c1f7842b3 100644
--- a/python/tvm/contrib/pipeline_executor.py
+++ b/python/tvm/contrib/pipeline_executor.py
@@ -157,7 +157,7 @@ def create(pipeline_mods, mod_config):
         mod = graph_executor.GraphModule(
             pipeline_mod["default"](pipeline_mods[pipeline_mod]["dev"])
         )
-        mods.append(mod.module)
+        mods.append(mod)

     submodule = PipelineModule(mods, json.dumps(mod_config))
     # submodule = PipelineModule(pipeline_mods, json.dumps(mod_config))
@@ -181,7 +181,7 @@ class PipelineModule(object):
     def __init__(self, modules, pipeline_config):
         mods = []
         for module in modules:
-            mods.append(module)
+            mods.append(module.module)

         pipelinecreate = tvm._ffi.get_global_func("tvm.pipeline_executor.create")
         assert pipelinecreate

```





---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-set-parameter-into-pipeline-module/11375/2) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/8e01753ed10136fbd50fe971dab348ff2064221d7c0790d9de9bc1e0636a78c0).

[Apache TVM Discuss] [Questions] How to set parameter into pipeline module

[popojames via Apache TVM Discuss <no...@discuss.tvm.ai>]

Thanks for replying,  I will try and keep an eye on the new patches. :slight_smile:





---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-set-parameter-into-pipeline-module/11375/3) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/b21e375afe7e487311617d371780511defb28e454df37ffe97007f8f14e48372).