You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tvm.apache.org by ANSHUMAN TRIPATHY via TVM Discuss <no...@discuss.tvm.ai> on 2020/05/11 14:42:51 UTC
[TVM Discuss] [Development/RFC] [DISCUSS] Module based Model Runtime
Interface
HI All, I was wondering, whether we can use [flatbuffer](https://google.github.io/flatbuffers/) for serializing params.
In that way we can customize the framework according to our suit as its opensource and it will be target agnostic.
I am working on a prototype currently. However i wanted to know all your expert opinions before hand.
Thanks
---
[Visit Topic](https://discuss.tvm.ai/t/discuss-module-based-model-runtime-interface/5025/63) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/ba6e92c385c98ec02c9716e86f8ee3a06fcc68fcd98f87b33740fa707d94274b).
[TVM Discuss] [Development/RFC] [DISCUSS] Module based Model Runtime
Interface
Posted by tqchen via TVM Discuss <no...@discuss.tvm.ai>.
Note that the parameter has to be loaded into DRAM, so there is no place where we could do partial weight load.
For memory limited scenarios like embedded devices, we would certainly need to go for a different solution, for example directly store weights in the rodata section to remove the need of serialization at all.
---
[Visit Topic](https://discuss.tvm.ai/t/discuss-module-based-model-runtime-interface/5025/66) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/7e2ad3dd1355dbf6afa57bd22eee5770b92f4bd1ec8beda0df2e7a3567b7e857).
[TVM Discuss] [Development/RFC] [DISCUSS] Module based Model Runtime
Interface
Posted by ANSHUMAN TRIPATHY via TVM Discuss <no...@discuss.tvm.ai>.
@tqchen: Thank you very much for your enlightening response!
I agree it will introduce an additional layer, but it may have an additional performance benefit as well even when the store is for simple objects with Flatbuffer or more precisely Flexbuffers used. I was thinking of a scenario when we intend to load only partial memory because of system memory limit, in that case this can be helpful. Just a thought :slightly_smiling_face:.
I just have a basic question here, currently Relay Build emits Params as Map, what format it should be saved and offered to user for future reuse?(Note: I am aware of save_dict api)
---
[Visit Topic](https://discuss.tvm.ai/t/discuss-module-based-model-runtime-interface/5025/65) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/97b3d8bc6330144012a68ad8b389d98fd9a26a177160d5dbc91d7d7e5925a050).
[TVM Discuss] [Development/RFC] [DISCUSS] Module based Model Runtime
Interface
Posted by tqchen via TVM Discuss <no...@discuss.tvm.ai>.
It would be helpful to ask why and why not when introducing new dependencies. See some of the examples in the design decision above. Flatbuffer coould be useful when we need to serialize a complicated set of objects, but also introduces an additional layer of abstraction.
Given that we are only dealing with a limited spec and current params serialization is already target agnostic, we do not need to introduce another layer of abstraction and dependency in this case.
---
[Visit Topic](https://discuss.tvm.ai/t/discuss-module-based-model-runtime-interface/5025/64) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/daafe152d1d2aa52b1795893c4dd80e51a8bcd53b62595d8d8fedd49989da74f).