You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/11/08 17:41:49 UTC
[GitHub] [tvm-rfcs] areusch commented on a change in pull request #43: [RFC][TVMC] Add support for micro targets
areusch commented on a change in pull request #43:
URL: https://github.com/apache/tvm-rfcs/pull/43#discussion_r744951442
##########
File path: rfcs/0021-tvmc_add_support_for_micro_targets.md
##########
@@ -0,0 +1,461 @@
+- Feature Name: tvmc_add_support_for_micro_targets
+- Start Date: 2021-10-09
+- RFC PR: [apache/tvm-rfcs#0000](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+
+This RFC obsoletes RFC **[0]**.
+
+This RFC depends on RFC **[1]**.
+
+This RFC is about how TVMC (TVM CLI tool) can be extended to support microTVM
+targets, considering the variety of platforms supported by microTVM, like Zephyr
+and Arduino, and also considering future platforms, taking into account the use
+of custom/adhoc platforms provided by users at their convenience.
+
+The interface here proposed relies on the new Project API available on microTVM.
+
+PR [9229](https://github.com/apache/tvm/pull/9229) implements the interface in
+question **[2]**.
+
+**[0]** [RFC] TVMC: Add support for µTVM -- https://discuss.tvm.apache.org/t/rfc-tvmc-add-support-for-tvm/9049
+
+**[1]** [RFC][Project API] Extend metadata in ProjectOption -- https://github.com/apache/tvm-rfcs/blob/main/rfcs/0020-project_api_extend_metadata.md
+
+**[2]** [TVMC] Add new micro context -- https://github.com/apache/tvm/pull/9229
+
+# Motivation
+
+Currently if a microTVM user wants to compile, build, flash, and run a model for
+a micro target available on a supported microTVM platform -- like Zephyr and
+Arduino -- it's necessary to drive the process end-to-end via Python scripts.
+This is ok when creating tests and developing or experimenting with new microTVM
+code, for example, but for some users that just want to try a new model of
+interest on a micro target that way can be slow.
+
+TVMC is a command-line driver for TVM (`tvmc`) that allows users to control the
+compilation and execution of models on a plethora of targets, using only the
+CLI, however it currently does not support microTVM targets, so it can not drive
+a similar workflow (compile, run, etc) on microcontrollers.
+
+This RFC proposes to extend TVMC to support the microTVM targets and platforms
+so one can easily use `tvmc` to **compile**, **build** (a device image with the
+compiled model of interest), **flash**, and **run** a model on a micro target
+using only a command-line interface.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Micro targets have unique requirements which justifies the creation of a new
+context in `tvmc` called `micro` to accommodate them.
+
+For instance, a given micro target can exist in more than one supported
+platform, like the STM32F746 MCU that can be supported by `Zephyr` and by
+`FreeRTOS`. Hence a platform must be selected in addition to the MCU and/or the
+board.
+
+Moreover, the build and flash steps, although might not be totally exclusive on
+micro targets, are quite specific (like using the toolchain and the flash tool
+supported by the platform), so much it justifies them to exist inside the new
+micro context, i.e. under `tvmc micro`.
+
+In that sense, the following subcommands are available under `tvmc micro`:
+
+**1. create-project**
+
+**2. build**
+
+**3. flash**
+
+`create-project` purpose is for creating a project directory based on **(a)** an
+already compiled model kept in a Model Library Format (MLF) archive and **(b)**
+a template associated to a platform, like Zephyr, allowing one to specify
+several options specific to the selected platform, like the board name and extra
+files to be included in the project directory to will be created based on the
+template directory. For convenience, an alias (`create`) will exist for this
+subcommand.
+
+`build` can be used for building a firmware image ready to be flashed to the
+board specified. For example, if Zephyr platform is selected build a
+`zephyr.{elf,hex}` image. Pertinent options for 'build' given the platform
+selected as an option under 'build' will become available.
+
+`flash` can be used for effectively flashing, to the selected board, the
+built firmware. Again, pertinent options for 'flash' context considering the
+platform selected will be made available for the user, like for example options
+regarding the serial port number to be used when flashing the image to the
+board.
+
+`build` and `flash` subcommads are intended to help with debugging inference
+flows built on top of Project API infrastructure. Hence they’re not intended to
+replace the user’s workflow (e.g. users can still build and flash independently
+of these subcommands).
+
+For all these subcommads (1, 2, and 3) the platform template can be selected
+among the default platforms (`zephyr` and `arduino`) or can be a custom one, in
+that case the platform template must be selected instead. Upon selecting the
+template platform option `-d TEMPLATE_DIR` will become available and one can so
+specify the adhoc template dir (which must contain an adhoc Project API server
+as per the Project API specification).
+
+A **key-value pair** is used to specify the options specific to the platforms.
+If the option is of type 'bool' the values available are 'true' and 'false'.
+Details on project option types are described in [RFC-0020](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0020-project_api_extend_metadata.md).
+
+An example for that interface (using Zephyr) follows:
+
+1. `create-project` subcommand:
+
+```bash
+$ tvmc micro
+usage: tvmc micro [-h] {create-project,build,flash} ...
+tvmc micro: error: the following arguments are required: subcommand
+
+$ tvmc micro -h
+usage: tvmc micro [-h] {create-project,build,flash} ...
+
+optional arguments:
+ -h, --help show this help message and exit
+
+subcommands:
+ {create-project,build,flash}
+ create-project create a project template of a given type or given a template dir.
+ build build a project dir, generally creating an image to be flashed, e.g. zephyr.elf.
+ flash flash the built image on a given micro target.
+
+$ tvmc micro create-project
+usage: tvmc micro create-project [-h] [-f] PROJECT_DIR MLF {zephyr,arduino,template} ...
+tvmc micro create-project: error: the following arguments are required: PROJECT_DIR, MLF, platform
+
+$ tvmc micro create-project /tmp/x100 ./sine_model.tar zephyr
+usage: tvmc micro create-project PROJECT_DIR MLF zephyr [--list-options] -o OPTION=VALUE [OPTION=VALUE ...]
+tvmc micro create-project PROJECT_DIR MLF zephyr: error: the following arguments are required: -o
+
+$ tvmc micro create-project /tmp/x100 ./sine_model.tar zephyr --list-options
+usage: tvmc micro create-project PROJECT_DIR MLF zephyr [--list-options] -o OPTION=VALUE [OPTION=VALUE ...]
+
+optional arguments:
+ --list-options show all options/values for selected platforms/template.
+ -o OPTION=VALUE [OPTION=VALUE ...]
+ extra_files_tar=EXTRA_FILES_TAR
+ if given, during generate_project, uncompress the tarball at this path into the project dir.
+
+ project_type={aot_demo, host_driven}
+ type of project to generate. (required)
+
+ west_cmd=WEST_CMD
+ path to the west tool. If given, supersedes both the zephyr_base option and ZEPHYR_BASE environment variable.
+
+ zephyr_board={mimxrt1050_evk, mps2_an521, nrf5340dk_nrf5340_cpuapp,
+ nucleo_f746zg, nucleo_l4r5zi, qemu_cortex_r5, qemu_riscv32,
+ qemu_riscv64, qemu_x86, stm32f746g_disco}
+ name of the Zephyr board to build for. (required)
+
+ config_main_stack_size=CONFIG_MAIN_STACK_SIZE
+ sets CONFIG_MAIN_STACK_SIZE for Zephyr board.
+
+$ tvmc micro create-project /tmp/x200 ./sine_model.tar zephyr -o zephyr_board=stm32f746g_disco project_type=host_driven
+$
+```
+
+2. `build` subcommand:
+
+```bash
+$ tvmc micro build
+usage: tvmc micro build [-h] [-f] PROJECT_DIR {zephyr,arduino,template} ...
+tvmc micro build: error: the following arguments are required: PROJECT_DIR, platform
+
+$ tvmc micro build -h
+usage: tvmc micro build [-h] [-f] PROJECT_DIR {zephyr,arduino,template} ...
+
+positional arguments:
+ PROJECT_DIR Project dir to build.
+
+optional arguments:
+ -h, --help show this help message and exit
+ -f, --force Force rebuild.
+
+platforms:
+ {zephyr,arduino,template}
+ you must selected a platform from the list. You can pass '-h' for a selected platform to list its options.
+ zephyr select Zephyr platform.
+ arduino select Arduino platform.
+ template select an adhoc template.
+
+$ tvmc micro build /tmp/x200 zephyr
+usage: tvmc micro build PROJECT_DIR zephyr [--list-options] -o OPTION=VALUE [OPTION=VALUE ...]
+tvmc micro build PROJECT_DIR zephyr: error: the following arguments are required: -o
+
+$ tvmc micro build /tmp/x200 zephyr --list-options
+usage: tvmc micro build PROJECT_DIR zephyr [--list-options] -o OPTION=VALUE [OPTION=VALUE ...]
+
+optional arguments:
+ --list-options show all options/values for selected platforms/template.
+ -o OPTION=VALUE [OPTION=VALUE ...]
+ verbose={true, false}
+ run build with verbose output.
+
+ zephyr_base=ZEPHYR_BASE
+ path to the zephyr base directory.
+
+ zephyr_board={mimxrt1050_evk, mps2_an521, nrf5340dk_nrf5340_cpuapp,
+ nucleo_f746zg, nucleo_l4r5zi, qemu_cortex_r5, qemu_riscv32,
+ qemu_riscv64, qemu_x86, stm32f746g_disco}
+ name of the Zephyr board to build for. (required)
+
+$ tvmc micro build /tmp/x200 zephyr -o zephyr_board=stm32f746g_disco
+$
+```
+
+3. `flash` subcommand:
+
+```bash
+$ tvmc micro flash
+usage: tvmc micro flash [-h] PROJECT_DIR {zephyr,arduino,template} ...
+tvmc micro flash: error: the following arguments are required: PROJECT_DIR, platform
+
+$ tvmc micro flash -h
+usage: tvmc micro flash [-h] PROJECT_DIR {zephyr,arduino,template} ...
+
+positional arguments:
+ PROJECT_DIR Project dir with a image built.
+
+optional arguments:
+ -h, --help show this help message and exit
+
+platforms:
+ {zephyr,arduino,template}
+ you must selected a platform from the list. You can pass '-h' for a selected platform to list its options.
+ zephyr select Zephyr platform.
+ arduino select Arduino platform.
+ template select an adhoc template.
+
+$ tvmc micro flash /tmp/x200 zephyr
+usage: tvmc micro flash PROJECT_DIR zephyr [--list-options] -o OPTION=VALUE [OPTION=VALUE ...]
+tvmc micro flash PROJECT_DIR zephyr: error: the following arguments are required: -o
+
+$ tvmc micro flash /tmp/x200 zephyr --list-options
+usage: tvmc micro flash PROJECT_DIR zephyr [--list-options] -o OPTION=VALUE [OPTION=VALUE ...]
+
+optional arguments:
+ --list-options show all options/values for selected platforms/template.
+ -o OPTION=VALUE [OPTION=VALUE ...]
+ zephyr_board={mimxrt1050_evk, mps2_an521, nrf5340dk_nrf5340_cpuapp,
+ nucleo_f746zg, nucleo_l4r5zi, qemu_cortex_r5, qemu_riscv32,
+ qemu_riscv64, qemu_x86, stm32f746g_disco}
+ name of the Zephyr board to build for. (required)
+
+$ tvmc micro flash /tmp/x200 zephyr -o zephyr_board=stm32f746g_disco
+$
+```
+
+## Running a model
+[running-a-model]: #running-a-model
+
+As shown above by using `tvmc micro` subcommands `create-project`, `build`, and
+`flash` it's possible to get a model flashed and ready to run on a micro target.
+
+Running a model is naturally common to all targets (not exclusive on MCUs), so
+commands and options for running a model on microcontrollers are accommodated
+under the existing `tvmc run` command. To select a micro target the device
+'micro' needs to be specified with `--device micro`. When it happens `PATH`
+argument can be used not to select a compiled model module file (like currently
+it happens on non-micro targets) but to specify the project directory used for
+flashing the model of interest:
+
+```bash
+
+$ tvmc run --device micro -h
+usage: tvmc run [-h] [--device {cpu,cuda,cl,metal,vulkan,rocm,micro}] [--fill-mode {zeros,ones,random}] [-i INPUTS] [-o OUTPUTS] [--print-time] [--print-top N] [--profile] [--repeat N] [--number N]
+ [--rpc-key RPC_KEY] [--rpc-tracker RPC_TRACKER]
+ PATH
+
+positional arguments:
+ PATH path to the compiled module file or to the project directory (micro devices only).
+
+optional arguments:
+ -h, --help show this help message and exit
+ --device {cpu,cuda,cl,metal,vulkan,rocm,micro}
+ target device to run the compiled module. Defaults to 'cpu'
+
+[...]
+```
+
+Once both `micro` device and project directory are specified `--list-options`
Review comment:
would `--help` not do this? it's okay to go with this route, just curious
##########
File path: rfcs/0021-tvmc_add_support_for_micro_targets.md
##########
@@ -0,0 +1,461 @@
+- Feature Name: tvmc_add_support_for_micro_targets
+- Start Date: 2021-10-09
+- RFC PR: [apache/tvm-rfcs#0000](https://github.com/apache/tvm-rfcs/pull/0000)
+- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+
+This RFC obsoletes RFC **[0]**.
+
+This RFC depends on RFC **[1]**.
+
+This RFC is about how TVMC (TVM CLI tool) can be extended to support microTVM
+targets, considering the variety of platforms supported by microTVM, like Zephyr
+and Arduino, and also considering future platforms, taking into account the use
+of custom/adhoc platforms provided by users at their convenience.
+
+The interface here proposed relies on the new Project API available on microTVM.
+
+PR [9229](https://github.com/apache/tvm/pull/9229) implements the interface in
+question **[2]**.
+
+**[0]** [RFC] TVMC: Add support for µTVM -- https://discuss.tvm.apache.org/t/rfc-tvmc-add-support-for-tvm/9049
+
+**[1]** [RFC][Project API] Extend metadata in ProjectOption -- https://github.com/apache/tvm-rfcs/blob/main/rfcs/0020-project_api_extend_metadata.md
+
+**[2]** [TVMC] Add new micro context -- https://github.com/apache/tvm/pull/9229
+
+# Motivation
+
+Currently if a microTVM user wants to compile, build, flash, and run a model for
+a micro target available on a supported microTVM platform -- like Zephyr and
+Arduino -- it's necessary to drive the process end-to-end via Python scripts.
+This is ok when creating tests and developing or experimenting with new microTVM
+code, for example, but for some users that just want to try a new model of
+interest on a micro target that way can be slow.
+
+TVMC is a command-line driver for TVM (`tvmc`) that allows users to control the
+compilation and execution of models on a plethora of targets, using only the
+CLI, however it currently does not support microTVM targets, so it can not drive
+a similar workflow (compile, run, etc) on microcontrollers.
+
+This RFC proposes to extend TVMC to support the microTVM targets and platforms
+so one can easily use `tvmc` to **compile**, **build** (a device image with the
+compiled model of interest), **flash**, and **run** a model on a micro target
+using only a command-line interface.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Micro targets have unique requirements which justifies the creation of a new
+context in `tvmc` called `micro` to accommodate them.
+
+For instance, a given micro target can exist in more than one supported
+platform, like the STM32F746 MCU that can be supported by `Zephyr` and by
+`FreeRTOS`. Hence a platform must be selected in addition to the MCU and/or the
+board.
+
+Moreover, the build and flash steps, although might not be totally exclusive on
+micro targets, are quite specific (like using the toolchain and the flash tool
+supported by the platform), so much it justifies them to exist inside the new
+micro context, i.e. under `tvmc micro`.
+
+In that sense, the following subcommands are available under `tvmc micro`:
+
+**1. create-project**
+
+**2. build**
+
+**3. flash**
+
+`create-project` purpose is for creating a project directory based on **(a)** an
+already compiled model kept in a Model Library Format (MLF) archive and **(b)**
+a template associated to a platform, like Zephyr, allowing one to specify
+several options specific to the selected platform, like the board name and extra
+files to be included in the project directory to will be created based on the
+template directory. For convenience, an alias (`create`) will exist for this
+subcommand.
+
+`build` can be used for building a firmware image ready to be flashed to the
+board specified. For example, if Zephyr platform is selected build a
+`zephyr.{elf,hex}` image. Pertinent options for 'build' given the platform
+selected as an option under 'build' will become available.
+
+`flash` can be used for effectively flashing, to the selected board, the
+built firmware. Again, pertinent options for 'flash' context considering the
+platform selected will be made available for the user, like for example options
+regarding the serial port number to be used when flashing the image to the
+board.
+
+`build` and `flash` subcommads are intended to help with debugging inference
+flows built on top of Project API infrastructure. Hence they’re not intended to
+replace the user’s workflow (e.g. users can still build and flash independently
+of these subcommands).
+
+For all these subcommads (1, 2, and 3) the platform template can be selected
+among the default platforms (`zephyr` and `arduino`) or can be a custom one, in
+that case the platform template must be selected instead. Upon selecting the
+template platform option `-d TEMPLATE_DIR` will become available and one can so
+specify the adhoc template dir (which must contain an adhoc Project API server
+as per the Project API specification).
+
+A **key-value pair** is used to specify the options specific to the platforms.
+If the option is of type 'bool' the values available are 'true' and 'false'.
+Details on project option types are described in [RFC-0020](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0020-project_api_extend_metadata.md).
+
+An example for that interface (using Zephyr) follows:
+
+1. `create-project` subcommand:
+
+```bash
+$ tvmc micro
+usage: tvmc micro [-h] {create-project,build,flash} ...
+tvmc micro: error: the following arguments are required: subcommand
+
+$ tvmc micro -h
+usage: tvmc micro [-h] {create-project,build,flash} ...
+
+optional arguments:
+ -h, --help show this help message and exit
+
+subcommands:
+ {create-project,build,flash}
+ create-project create a project template of a given type or given a template dir.
+ build build a project dir, generally creating an image to be flashed, e.g. zephyr.elf.
+ flash flash the built image on a given micro target.
+
+$ tvmc micro create-project
+usage: tvmc micro create-project [-h] [-f] PROJECT_DIR MLF {zephyr,arduino,template} ...
+tvmc micro create-project: error: the following arguments are required: PROJECT_DIR, MLF, platform
+
+$ tvmc micro create-project /tmp/x100 ./sine_model.tar zephyr
+usage: tvmc micro create-project PROJECT_DIR MLF zephyr [--list-options] -o OPTION=VALUE [OPTION=VALUE ...]
+tvmc micro create-project PROJECT_DIR MLF zephyr: error: the following arguments are required: -o
+
+$ tvmc micro create-project /tmp/x100 ./sine_model.tar zephyr --list-options
+usage: tvmc micro create-project PROJECT_DIR MLF zephyr [--list-options] -o OPTION=VALUE [OPTION=VALUE ...]
+
+optional arguments:
+ --list-options show all options/values for selected platforms/template.
+ -o OPTION=VALUE [OPTION=VALUE ...]
+ extra_files_tar=EXTRA_FILES_TAR
+ if given, during generate_project, uncompress the tarball at this path into the project dir.
+
+ project_type={aot_demo, host_driven}
+ type of project to generate. (required)
+
+ west_cmd=WEST_CMD
+ path to the west tool. If given, supersedes both the zephyr_base option and ZEPHYR_BASE environment variable.
+
+ zephyr_board={mimxrt1050_evk, mps2_an521, nrf5340dk_nrf5340_cpuapp,
+ nucleo_f746zg, nucleo_l4r5zi, qemu_cortex_r5, qemu_riscv32,
+ qemu_riscv64, qemu_x86, stm32f746g_disco}
+ name of the Zephyr board to build for. (required)
+
+ config_main_stack_size=CONFIG_MAIN_STACK_SIZE
+ sets CONFIG_MAIN_STACK_SIZE for Zephyr board.
+
+$ tvmc micro create-project /tmp/x200 ./sine_model.tar zephyr -o zephyr_board=stm32f746g_disco project_type=host_driven
+$
+```
+
+2. `build` subcommand:
+
+```bash
+$ tvmc micro build
+usage: tvmc micro build [-h] [-f] PROJECT_DIR {zephyr,arduino,template} ...
+tvmc micro build: error: the following arguments are required: PROJECT_DIR, platform
+
+$ tvmc micro build -h
+usage: tvmc micro build [-h] [-f] PROJECT_DIR {zephyr,arduino,template} ...
+
+positional arguments:
+ PROJECT_DIR Project dir to build.
+
+optional arguments:
+ -h, --help show this help message and exit
+ -f, --force Force rebuild.
+
+platforms:
+ {zephyr,arduino,template}
+ you must selected a platform from the list. You can pass '-h' for a selected platform to list its options.
+ zephyr select Zephyr platform.
+ arduino select Arduino platform.
+ template select an adhoc template.
+
+$ tvmc micro build /tmp/x200 zephyr
+usage: tvmc micro build PROJECT_DIR zephyr [--list-options] -o OPTION=VALUE [OPTION=VALUE ...]
+tvmc micro build PROJECT_DIR zephyr: error: the following arguments are required: -o
+
+$ tvmc micro build /tmp/x200 zephyr --list-options
+usage: tvmc micro build PROJECT_DIR zephyr [--list-options] -o OPTION=VALUE [OPTION=VALUE ...]
+
+optional arguments:
+ --list-options show all options/values for selected platforms/template.
+ -o OPTION=VALUE [OPTION=VALUE ...]
+ verbose={true, false}
+ run build with verbose output.
+
+ zephyr_base=ZEPHYR_BASE
+ path to the zephyr base directory.
+
+ zephyr_board={mimxrt1050_evk, mps2_an521, nrf5340dk_nrf5340_cpuapp,
+ nucleo_f746zg, nucleo_l4r5zi, qemu_cortex_r5, qemu_riscv32,
+ qemu_riscv64, qemu_x86, stm32f746g_disco}
+ name of the Zephyr board to build for. (required)
+
+$ tvmc micro build /tmp/x200 zephyr -o zephyr_board=stm32f746g_disco
+$
+```
+
+3. `flash` subcommand:
+
+```bash
+$ tvmc micro flash
+usage: tvmc micro flash [-h] PROJECT_DIR {zephyr,arduino,template} ...
+tvmc micro flash: error: the following arguments are required: PROJECT_DIR, platform
+
+$ tvmc micro flash -h
+usage: tvmc micro flash [-h] PROJECT_DIR {zephyr,arduino,template} ...
+
+positional arguments:
+ PROJECT_DIR Project dir with a image built.
+
+optional arguments:
+ -h, --help show this help message and exit
+
+platforms:
+ {zephyr,arduino,template}
+ you must selected a platform from the list. You can pass '-h' for a selected platform to list its options.
+ zephyr select Zephyr platform.
+ arduino select Arduino platform.
+ template select an adhoc template.
+
+$ tvmc micro flash /tmp/x200 zephyr
+usage: tvmc micro flash PROJECT_DIR zephyr [--list-options] -o OPTION=VALUE [OPTION=VALUE ...]
+tvmc micro flash PROJECT_DIR zephyr: error: the following arguments are required: -o
Review comment:
just curious: why is `-o` a required arg? seems it should be always optional (i agree zephyr_board is required for the zephyr `microtvm_api_server.py`, but as an API-level concept, it shouldn't always be required to specify `-o`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org