You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/07/01 00:11:21 UTC
[GitHub] [tvm] areusch opened a new pull request #8380: [µTVM] Commit Project API infrastructure
areusch opened a new pull request #8380:
URL: https://github.com/apache/tvm/pull/8380
This PR implements the [Project API RFC](https://github.com/apache/tvm-rfcs/pull/8), a plugin-like infrastructure that allows TVM to interact with build tools for third-party platforms. By implementing the Project API interface for a platform, developers can run core TVM workflows such as host-driven inference and AutoTVM without needing to build the full TVM library on the platform. This is initially targeted at microTVM use cases, where platforms don't have the OS facilities nor the flash space to run a full TVM C++ runtime nor a Python interpreter.
More discussion is available in the RFC. Implementations are provided here for Zephyr and for the POSIX subprocess test platform.
@mehrdadh @guberti @tqchen @jroesch @tkonolige @csullivan @leandron @u99127 @Mousius @giuseros @gromero @stoa @hogepodge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-887922573
@mehrdadh @gromero please take another look!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r677865658
##########
File path: apps/bundle_deploy/crt_config/crt_config.h
##########
@@ -35,9 +35,9 @@
/*! Maximum supported arguments in generated functions */
#define TVM_CRT_MAX_ARGS 10
/*! Maximum supported string length in dltype, e.g. "int8", "int16", "float32" */
-#define TVM_CRT_STRLEN_DLTYPE 10
+#define TVM_CRT_MAX_STRLEN_DLTYPE 10
Review comment:
good catch, i agree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-893886220
@areusch Thanks a lot for addressing all the issues and concerns. LGTM!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r679604499
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,692 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import collections
+import collections.abc
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.abc.Mapping):
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}"
+ )
+
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_l4r5zi": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(f'No attached boards recognized by {" ".join(nrfjprog_args)}')
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options["nrfjprog_snr"] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: " f'{", ".join(boards)}'
+ )
+
+ if str(options["nrfjprog_snr"]) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options["nrfjprog_snr"]]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_TYPES = []
+if IS_TEMPLATE:
+ for d in (API_SERVER_DIR / "src").iterdir():
+ if d.is_dir():
+ PROJECT_TYPES.append(d.name)
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption(
+ "extra_files",
+ help="If given, during generate_project, uncompress the tarball at this path into the project dir",
+ ),
+ server.ProjectOption(
+ "gdbserver_port", help=("If given, port number to use when running the local gdbserver")
+ ),
+ server.ProjectOption(
+ "nrfjprog_snr",
+ help=(
+ "When used with nRF targets, serial # of the " "attached board to use, from nrfjprog"
+ ),
+ ),
+ server.ProjectOption(
+ "openocd_serial",
+ help=("When used with OpenOCD targets, serial # of the " "attached board to use"),
+ ),
+ server.ProjectOption(
+ "project_type",
+ help="Type of project to generate.",
+ choices=tuple(PROJECT_TYPES),
+ ),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption(
+ "west_cmd",
+ help=(
+ "Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable."
+ ),
+ ),
+ server.ProjectOption("zephyr_base", help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self, tvm_version):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path=""
+ if IS_TEMPLATE
+ else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS,
+ )
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def _create_prj_conf(self, project_dir, options):
+ with open(project_dir / "prj.conf", "w") as f:
+ f.write(
+ "# For UART used from main().\n"
+ "CONFIG_RING_BUFFER=y\n"
+ "CONFIG_UART_CONSOLE=n\n"
+ "CONFIG_UART_INTERRUPT_DRIVEN=y\n"
+ "\n"
+ )
+ f.write("# For TVMPlatformAbort().\n" "CONFIG_REBOOT=y\n" "\n")
+
+ if True: # options["project_type"] == "host_driven":
+ f.write("# For RPC server C++ bindings.\n" "CONFIG_CPLUSPLUS=y\n" "\n")
+
+ f.write("# For math routines\n" "CONFIG_NEWLIB_LIBC=y\n" "\n")
+
+ if self._has_fpu(options["zephyr_board"]):
+ f.write("# For models with floating point.\n" "CONFIG_FPU=y\n" "\n")
+
+ main_stack_size = None
+ if self._is_qemu(options) and options["project_type"] == "host_driven":
Review comment:
@mehrdadh actually, I've tested the previous version, so I'm re-testing it on disco. Hence it's good to test as you said with the 4 additional boards you have on hand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r662677398
##########
File path: python/tvm/micro/project.py
##########
@@ -0,0 +1,97 @@
+"""Defines glue wrappers around the Project API which mate to TVM interfaces."""
+
+from ..contrib import utils
+from .build import get_standalone_crt_dir
+from .model_library_format import export_model_library_format
+from .project_api import client
+from .transport import Transport, TransportTimeouts
+
+
+class ProjectTransport(Transport):
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._timeouts = None
+
+ def timeouts(self):
+ assert self._timeouts is not None, "Transport not yet opened"
+ return self._timeouts
+
+ def open(self):
+ reply = self._client.connect_transport(self._options)
+ self._timeouts = TransportTimeouts(**reply["timeouts"])
+
+ def close(self):
+ self._client.disconnect_transport()
+
+ def write(self, data, timeout_sec):
+ return self._client.write_transport(data, timeout_sec)["bytes_written"]
+
+ def read(self, n, timeout_sec):
+ return self._client.read_transport(n, timeout_sec)["data"]
+
+
+class TemplateProjectError(Exception):
+ """Raised when the Project API server given to GeneratedProject reports is_template=True."""
+
+
+class GeneratedProject:
+ """Defines a glue interface to interact with a generated project through the API server."""
+
+ @classmethod
+ def from_directory(cls, project_dir, options):
+ return cls(client.instantiate_from_dir(project_dir), options)
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._info = self._client.server_info_query()
+ if self._info['is_template']:
+ raise TemplateProjectError()
+
Review comment:
I think `server_info_query()` must be exposed as well just like `build()` and friends, like:
```diff --git a/python/tvm/micro/project.py b/python/tvm/micro/project.py
index 096db4bdd..fb6e0b94b 100644
--- a/python/tvm/micro/project.py
+++ b/python/tvm/micro/project.py
@@ -51,6 +51,9 @@ class GeneratedProject:
if self._info['is_template']:
raise TemplateProjectError()
+ def server_info_query(self):
+ return self._client.server_info_query()
+
def build(self):
self._client.build(self._options)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-888658001
@areusch Fixes in `https://github.com/apache/tvm/pull/8380/commits/0d0c504a158ac9e03c301b88a5ca1127c57bd06e` are good. I think the only remaining issue is that `prj.conf` generation mechanism currently doesn't account for boards that don't have a FPU, so `CONFIG_FPU=y` (like the `mps2_an521` board). So I think we need to fix it because it's a regression of https://github.com/apache/tvm/pull/8055. Otherwise I think it's pretty much ready to land once CI is happy too :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-892996827
tested offline against nrf5340 and stm32f746xx_disco; waiting for CI to be fixed to retrigger. @gromero @mehrdadh can you take a look when you have a minute?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leandron commented on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
leandron commented on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-894363382
I'm happy for anyone to merge this once the conflicting files are solved and CI is green again. cc @tmoreau89
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r677788445
##########
File path: python/tvm/micro/project.py
##########
@@ -0,0 +1,97 @@
+"""Defines glue wrappers around the Project API which mate to TVM interfaces."""
+
+from ..contrib import utils
+from .build import get_standalone_crt_dir
+from .model_library_format import export_model_library_format
+from .project_api import client
+from .transport import Transport, TransportTimeouts
+
+
+class ProjectTransport(Transport):
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._timeouts = None
+
+ def timeouts(self):
+ assert self._timeouts is not None, "Transport not yet opened"
+ return self._timeouts
+
+ def open(self):
+ reply = self._client.connect_transport(self._options)
+ self._timeouts = TransportTimeouts(**reply["timeouts"])
+
+ def close(self):
+ self._client.disconnect_transport()
+
+ def write(self, data, timeout_sec):
+ return self._client.write_transport(data, timeout_sec)["bytes_written"]
+
+ def read(self, n, timeout_sec):
+ return self._client.read_transport(n, timeout_sec)["data"]
+
+
+class TemplateProjectError(Exception):
+ """Raised when the Project API server given to GeneratedProject reports is_template=True."""
+
+
+class GeneratedProject:
+ """Defines a glue interface to interact with a generated project through the API server."""
+
+ @classmethod
+ def from_directory(cls, project_dir, options):
+ return cls(client.instantiate_from_dir(project_dir), options)
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._info = self._client.server_info_query()
+ if self._info['is_template']:
+ raise TemplateProjectError()
+
Review comment:
@areusch I thought of this code as a convenient wrapper around `TemplateProject` class and, ultimately, around project_api.client module. Hence in the use case where a program wants to simply query some info only about the template project (without generating one) it would use the `server_info_query()` method. For instance, with the most recent code there is now a new `options` '`project_type` (which can be currently `host_driven` or `aot_driven`. TVMC create-project command would need to accept an argument specifying which one it can take. Since Client API detects it automatically based on what's in `src/`, it would be matter of querying the `project_type`s available by using `TemplateProject.server_info_query` to show the user the options available - we would need also create an associated wrapper like `tvmc.micro.project.template_project` On a second run the user woulds already know the right option and pass to TVMC create-project command, which by its turn would that tim
e call `tvm.micro.project.create_project`.
Maybe I've misunderstood some project.py role. Let me know if it's not clear the use case I'm thinking of.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r679555115
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,692 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import collections
+import collections.abc
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.abc.Mapping):
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}"
+ )
+
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_l4r5zi": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(f'No attached boards recognized by {" ".join(nrfjprog_args)}')
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options["nrfjprog_snr"] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: " f'{", ".join(boards)}'
+ )
+
+ if str(options["nrfjprog_snr"]) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options["nrfjprog_snr"]]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_TYPES = []
+if IS_TEMPLATE:
+ for d in (API_SERVER_DIR / "src").iterdir():
+ if d.is_dir():
+ PROJECT_TYPES.append(d.name)
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption(
+ "extra_files",
+ help="If given, during generate_project, uncompress the tarball at this path into the project dir",
+ ),
+ server.ProjectOption(
+ "gdbserver_port", help=("If given, port number to use when running the local gdbserver")
+ ),
+ server.ProjectOption(
+ "nrfjprog_snr",
+ help=(
+ "When used with nRF targets, serial # of the " "attached board to use, from nrfjprog"
+ ),
+ ),
+ server.ProjectOption(
+ "openocd_serial",
+ help=("When used with OpenOCD targets, serial # of the " "attached board to use"),
+ ),
+ server.ProjectOption(
+ "project_type",
+ help="Type of project to generate.",
+ choices=tuple(PROJECT_TYPES),
+ ),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption(
+ "west_cmd",
+ help=(
+ "Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable."
+ ),
+ ),
+ server.ProjectOption("zephyr_base", help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self, tvm_version):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path=""
+ if IS_TEMPLATE
+ else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS,
+ )
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def _create_prj_conf(self, project_dir, options):
+ with open(project_dir / "prj.conf", "w") as f:
+ f.write(
+ "# For UART used from main().\n"
+ "CONFIG_RING_BUFFER=y\n"
+ "CONFIG_UART_CONSOLE=n\n"
+ "CONFIG_UART_INTERRUPT_DRIVEN=y\n"
+ "\n"
+ )
+ f.write("# For TVMPlatformAbort().\n" "CONFIG_REBOOT=y\n" "\n")
+
+ if True: # options["project_type"] == "host_driven":
+ f.write("# For RPC server C++ bindings.\n" "CONFIG_CPLUSPLUS=y\n" "\n")
+
+ f.write("# For math routines\n" "CONFIG_NEWLIB_LIBC=y\n" "\n")
+
+ if self._has_fpu(options["zephyr_board"]):
+ f.write("# For models with floating point.\n" "CONFIG_FPU=y\n" "\n")
+
+ main_stack_size = None
+ if self._is_qemu(options) and options["project_type"] == "host_driven":
Review comment:
@mehrdadh the API server does work on physical boards, I've tested it on disco boards, for instance. `test_zephyr` and `test_zephyr_aot` defaults to use qemu as on the old API, but `prj.conf` generator will take care of generating the proper config file for physical boards too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r684484500
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,698 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import atexit
+import collections
+import collections.abc
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.abc.Mapping):
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}"
+ )
+
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_l4r5zi": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(f'No attached boards recognized by {" ".join(nrfjprog_args)}')
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options["nrfjprog_snr"] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: " f'{", ".join(boards)}'
+ )
+
+ if str(options["nrfjprog_snr"]) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options["nrfjprog_snr"]]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_TYPES = []
+if IS_TEMPLATE:
+ for d in (API_SERVER_DIR / "src").iterdir():
+ if d.is_dir():
+ PROJECT_TYPES.append(d.name)
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption(
+ "extra_files",
+ help="If given, during generate_project, uncompress the tarball at this path into the project dir",
+ ),
+ server.ProjectOption(
+ "gdbserver_port", help=("If given, port number to use when running the local gdbserver")
+ ),
+ server.ProjectOption(
+ "nrfjprog_snr",
+ help=(
+ "When used with nRF targets, serial # of the " "attached board to use, from nrfjprog"
+ ),
+ ),
+ server.ProjectOption(
+ "openocd_serial",
+ help=("When used with OpenOCD targets, serial # of the " "attached board to use"),
+ ),
+ server.ProjectOption(
+ "project_type",
+ help="Type of project to generate.",
+ choices=tuple(PROJECT_TYPES),
+ ),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption(
+ "west_cmd",
+ help=(
+ "Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable."
+ ),
+ ),
+ server.ProjectOption("zephyr_base", help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self, tvm_version):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path=""
+ if IS_TEMPLATE
+ else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS,
+ )
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def _create_prj_conf(self, project_dir, options):
+ with open(project_dir / "prj.conf", "w") as f:
+ f.write(
+ "# For UART used from main().\n"
+ "CONFIG_RING_BUFFER=y\n"
+ "CONFIG_UART_CONSOLE=n\n"
+ "CONFIG_UART_INTERRUPT_DRIVEN=y\n"
+ "\n"
+ )
+ f.write("# For TVMPlatformAbort().\n" "CONFIG_REBOOT=y\n" "\n")
+
+ if True: # options["project_type"] == "host_driven":
+ f.write("# For RPC server C++ bindings.\n" "CONFIG_CPLUSPLUS=y\n" "\n")
+
+ f.write("# For math routines\n" "CONFIG_NEWLIB_LIBC=y\n" "\n")
+
+ if self._has_fpu(options["zephyr_board"]):
+ f.write("# For models with floating point.\n" "CONFIG_FPU=y\n" "\n")
+
+ main_stack_size = None
+ if self._is_qemu(options) and options["project_type"] == "host_driven":
+ main_stack_size = 1536
+
+ # Set main stack size, if needed.
+ if main_stack_size is not None:
+ f.write(f"CONFIG_MAIN_STACK_SIZE={main_stack_size}\n")
+
+ f.write("# For random number generation.\n" "CONFIG_TEST_RANDOM_GENERATOR=y\n")
+
+ if self._is_qemu(options):
+ f.write("CONFIG_TIMER_RANDOM_GENERATOR=y\n")
+ else:
+ f.write("CONFIG_ENTROPY_GENERATOR=y\n")
+ f.write("\n")
+
+ API_SERVER_CRT_LIBS_TOKEN = "<API_SERVER_CRT_LIBS>"
+
+ CRT_LIBS_BY_PROJECT_TYPE = {
+ "host_driven": "microtvm_rpc_server microtvm_rpc_common common",
+ "aot_demo": "aot_executor memory microtvm_rpc_common common",
+ }
Review comment:
oops, fixed
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,698 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import atexit
+import collections
+import collections.abc
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.abc.Mapping):
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}"
+ )
+
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_l4r5zi": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(f'No attached boards recognized by {" ".join(nrfjprog_args)}')
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options["nrfjprog_snr"] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: " f'{", ".join(boards)}'
+ )
+
+ if str(options["nrfjprog_snr"]) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options["nrfjprog_snr"]]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_TYPES = []
+if IS_TEMPLATE:
+ for d in (API_SERVER_DIR / "src").iterdir():
+ if d.is_dir():
+ PROJECT_TYPES.append(d.name)
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption(
+ "extra_files",
+ help="If given, during generate_project, uncompress the tarball at this path into the project dir",
+ ),
+ server.ProjectOption(
+ "gdbserver_port", help=("If given, port number to use when running the local gdbserver")
+ ),
+ server.ProjectOption(
+ "nrfjprog_snr",
+ help=(
+ "When used with nRF targets, serial # of the " "attached board to use, from nrfjprog"
+ ),
+ ),
+ server.ProjectOption(
+ "openocd_serial",
+ help=("When used with OpenOCD targets, serial # of the " "attached board to use"),
+ ),
+ server.ProjectOption(
+ "project_type",
+ help="Type of project to generate.",
+ choices=tuple(PROJECT_TYPES),
+ ),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption(
+ "west_cmd",
+ help=(
+ "Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable."
+ ),
+ ),
+ server.ProjectOption("zephyr_base", help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self, tvm_version):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path=""
+ if IS_TEMPLATE
+ else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS,
+ )
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def _create_prj_conf(self, project_dir, options):
+ with open(project_dir / "prj.conf", "w") as f:
+ f.write(
+ "# For UART used from main().\n"
+ "CONFIG_RING_BUFFER=y\n"
+ "CONFIG_UART_CONSOLE=n\n"
+ "CONFIG_UART_INTERRUPT_DRIVEN=y\n"
+ "\n"
+ )
+ f.write("# For TVMPlatformAbort().\n" "CONFIG_REBOOT=y\n" "\n")
+
+ if True: # options["project_type"] == "host_driven":
Review comment:
good catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r677889527
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,533 @@
+import collections
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = '' if 'cwd' not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.Mapping):
+
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options, cmake_entries):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}")
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(
+ f'No attached boards recognized by {" ".join(nrfjprog_args)}'
+ )
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options['nrfjprog_snr'] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: "
+ f'{", ".join(boards)}'
+ )
+
+ if str(options['nrfjprog_snr']) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options['nrfjprog_snr']]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption("gdbserver_port", help=("If given, port number to use when running the "
+ "local gdbserver")),
+ server.ProjectOption("openocd_serial", help=("When used with OpenOCD targets, serial # of the "
+ "attached board to use")),
+ server.ProjectOption("nrfjprog_snr", help=("When used with nRF targets, serial # of the "
+ "attached board to use, from nrfjprog")),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption("west_cmd",
+ help=("Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable.")),
+ server.ProjectOption("zephyr_base",
+ help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path="" if IS_TEMPLATE else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS)
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def generate_project(self, model_library_format_path, standalone_crt_dir, project_dir, options):
+ project_dir = pathlib.Path(project_dir)
+ # Make project directory.
+ project_dir.mkdir()
+
+ # Copy ourselves to the generated project. TVM may perform further build steps on the generated project
+ # by launching the copy.
+ shutil.copy2(__file__, project_dir / os.path.basename(__file__))
+
+ # Place Model Library Format tarball in the special location, which this script uses to decide
+ # whether it's being invoked in a template or generated project.
+ project_model_library_format_tar_path = project_dir / MODEL_LIBRARY_FORMAT_RELPATH
+ shutil.copy2(model_library_format_path, project_model_library_format_tar_path)
+
+ shutil.copytree(API_SERVER_DIR / "boards", project_dir / "boards")
+
+ # Extract Model Library Format tarball.into <project_dir>/model.
+ extract_path = os.path.splitext(project_model_library_format_tar_path)[0]
+ with tarfile.TarFile(project_model_library_format_tar_path) as tf:
+ os.makedirs(extract_path)
+ tf.extractall(path=extract_path)
+
+ if self._is_qemu(options):
+ shutil.copytree(API_SERVER_DIR / "qemu-hack", project_dir / "qemu-hack")
+
+ # Populate CRT.
+ crt_path = project_dir / "crt"
+ crt_path.mkdir()
+ for item in self.CRT_COPY_ITEMS:
+ src_path = os.path.join(standalone_crt_dir, item)
+ dst_path = crt_path / item
+ if os.path.isdir(src_path):
+ shutil.copytree(src_path, dst_path)
+ else:
+ shutil.copy2(src_path, dst_path)
+
+ # Populate Makefile.
+ shutil.copy2(API_SERVER_DIR / "CMakeLists.txt", project_dir / "CMakeLists.txt")
+ shutil.copy2(API_SERVER_DIR / "prj.conf", project_dir / "prj.conf")
+
+ # Populate crt-config.h
+ crt_config_dir = project_dir / "crt_config"
+ crt_config_dir.mkdir()
+ shutil.copy2(API_SERVER_DIR / "crt_config" / "crt_config.h", crt_config_dir / "crt_config.h")
+
+ # Populate src/
+ src_dir = project_dir / "src"
+ src_dir.mkdir()
+ shutil.copy2(API_SERVER_DIR / "src" / "main.c", src_dir / "main.c")
+
+ def build(self, options):
+ BUILD_DIR.mkdir()
+
+ cmake_args = ["cmake", ".."]
+ if options.get("verbose"):
+ cmake_args.append("-DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE")
+
+ if options.get("zephyr_base"):
+ cmake_args.append(f"-DZEPHYR_BASE:STRING={options['zephyr_base']}")
+
+ cmake_args.append(f"-DBOARD:STRING={options['zephyr_board']}")
+
+ check_call(cmake_args, cwd=BUILD_DIR)
+
+ args = ["make", "-j2"]
+ if options.get("verbose"):
+ args.append("VERBOSE=1")
+ check_call(args, cwd=BUILD_DIR)
+
+ @classmethod
+ def _is_qemu(cls, options):
+ return "qemu" in options["zephyr_board"]
+
+ def flash(self, options):
+ if self._is_qemu(options):
+ return # NOTE: qemu requires no flash step--it is launched from open_transport.
+
+ zephyr_board = options["zephyr_board"]
+
+ # The nRF5340DK requires an additional `nrfjprog --recover` before each flash cycle.
+ # This is because readback protection is enabled by default when this device is flashed.
+ # Otherwise, flashing may fail with an error such as the following:
+ # ERROR: The operation attempted is unavailable due to readback protection in
+ # ERROR: your device. Please use --recover to unlock the device.
+ if zephyr_board.startswith("nrf5340dk") and _get_flash_runner() == "nrfjprog":
+ recover_args = ["nrfjprog", "--recover"]
+ recover_args.extend(_get_nrf_device_args(options))
+ self._subprocess_env.run(recover_args, cwd=build_dir)
Review comment:
resolved these problems!
##########
File path: python/tvm/micro/compiler.py
##########
@@ -1,361 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
@mehrdadh any thoughts here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r677788445
##########
File path: python/tvm/micro/project.py
##########
@@ -0,0 +1,97 @@
+"""Defines glue wrappers around the Project API which mate to TVM interfaces."""
+
+from ..contrib import utils
+from .build import get_standalone_crt_dir
+from .model_library_format import export_model_library_format
+from .project_api import client
+from .transport import Transport, TransportTimeouts
+
+
+class ProjectTransport(Transport):
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._timeouts = None
+
+ def timeouts(self):
+ assert self._timeouts is not None, "Transport not yet opened"
+ return self._timeouts
+
+ def open(self):
+ reply = self._client.connect_transport(self._options)
+ self._timeouts = TransportTimeouts(**reply["timeouts"])
+
+ def close(self):
+ self._client.disconnect_transport()
+
+ def write(self, data, timeout_sec):
+ return self._client.write_transport(data, timeout_sec)["bytes_written"]
+
+ def read(self, n, timeout_sec):
+ return self._client.read_transport(n, timeout_sec)["data"]
+
+
+class TemplateProjectError(Exception):
+ """Raised when the Project API server given to GeneratedProject reports is_template=True."""
+
+
+class GeneratedProject:
+ """Defines a glue interface to interact with a generated project through the API server."""
+
+ @classmethod
+ def from_directory(cls, project_dir, options):
+ return cls(client.instantiate_from_dir(project_dir), options)
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._info = self._client.server_info_query()
+ if self._info['is_template']:
+ raise TemplateProjectError()
+
Review comment:
@areusch I thought of this code as a convenient wrapper around `TemplateProject` class and, ultimately, around project_api.client module. Hence in the use case where a program wants to simply query some info only about the template project (without generating a project based on the template dir) it would use the `server_info_query()` method. For instance, with the most recent code there is now a new `options` '`project_type` (which can be currently `host_driven` or `aot_driven`. TVMC create-project command would need to accept an argument specifying which one it can take. Since Client API detects it automatically based on what's in `src/`, it would be matter of querying the `project_type`s available by using `TemplateProject.server_info_query` to show the user the options available - we would need also create an associated wrapper like `tvmc.micro.project.template_project` On a second run the user woulds already know the right option and pass to TVMC create-project command,
which by its turn would that time call `tvm.micro.project.create_project`.
Maybe I've misunderstood some project.py role. Let me know if it's not clear the use case I'm thinking of.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r677788445
##########
File path: python/tvm/micro/project.py
##########
@@ -0,0 +1,97 @@
+"""Defines glue wrappers around the Project API which mate to TVM interfaces."""
+
+from ..contrib import utils
+from .build import get_standalone_crt_dir
+from .model_library_format import export_model_library_format
+from .project_api import client
+from .transport import Transport, TransportTimeouts
+
+
+class ProjectTransport(Transport):
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._timeouts = None
+
+ def timeouts(self):
+ assert self._timeouts is not None, "Transport not yet opened"
+ return self._timeouts
+
+ def open(self):
+ reply = self._client.connect_transport(self._options)
+ self._timeouts = TransportTimeouts(**reply["timeouts"])
+
+ def close(self):
+ self._client.disconnect_transport()
+
+ def write(self, data, timeout_sec):
+ return self._client.write_transport(data, timeout_sec)["bytes_written"]
+
+ def read(self, n, timeout_sec):
+ return self._client.read_transport(n, timeout_sec)["data"]
+
+
+class TemplateProjectError(Exception):
+ """Raised when the Project API server given to GeneratedProject reports is_template=True."""
+
+
+class GeneratedProject:
+ """Defines a glue interface to interact with a generated project through the API server."""
+
+ @classmethod
+ def from_directory(cls, project_dir, options):
+ return cls(client.instantiate_from_dir(project_dir), options)
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._info = self._client.server_info_query()
+ if self._info['is_template']:
+ raise TemplateProjectError()
+
Review comment:
@areusch I thought of this code as a convenient wrapper around `TemplateProject` class and, ultimately around project_api.client module. Hence in the use case where a program wants to simply query some info only about the template project (without generating one) it would use the `server_info_query()` method. For instance, with the most recent code there is now a new `options` '`project_type` (which can be currently `host_driven` or `aot_driven`. TVMC create-project command would need to accept an argument specifying which one it can take. Since Client API detects it automatically based on what's in `src/`, it would be matter of querying the `project_type`s available by using `TemplateProject.server_info_query` to show the user the options available - we would need also create an associated wrapper like `tvmc.micro.project.template_project` On a second run the user woulds already know the right option and pass to TVMC create-project command, which by its turn would that time
call `tvm.micro.project.create_project`.
Maybe I've misunderstood some project.py role. Let me know if it's not clear the use case I'm thinking of.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mehrdadh commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
mehrdadh commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r679536728
##########
File path: python/tvm/micro/compiler.py
##########
@@ -1,361 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
@areusch I agree that since there's no official release we don't commit to support older stuff. However I recommend to make it more clear in the PR title/description that this PR is removing certain modules so if anyone has a problem could raise their voice.
The other thing is that you could announce it in the microtvm community meeting and get some feedback there. My concern is that people have internal projects depending on these features.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tmoreau89 commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
tmoreau89 commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r683866061
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,698 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import atexit
+import collections
+import collections.abc
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.abc.Mapping):
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}"
+ )
+
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_l4r5zi": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(f'No attached boards recognized by {" ".join(nrfjprog_args)}')
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options["nrfjprog_snr"] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: " f'{", ".join(boards)}'
+ )
+
+ if str(options["nrfjprog_snr"]) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options["nrfjprog_snr"]]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_TYPES = []
+if IS_TEMPLATE:
+ for d in (API_SERVER_DIR / "src").iterdir():
+ if d.is_dir():
+ PROJECT_TYPES.append(d.name)
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption(
+ "extra_files",
+ help="If given, during generate_project, uncompress the tarball at this path into the project dir",
+ ),
+ server.ProjectOption(
+ "gdbserver_port", help=("If given, port number to use when running the local gdbserver")
+ ),
+ server.ProjectOption(
+ "nrfjprog_snr",
+ help=(
+ "When used with nRF targets, serial # of the " "attached board to use, from nrfjprog"
+ ),
+ ),
+ server.ProjectOption(
+ "openocd_serial",
+ help=("When used with OpenOCD targets, serial # of the " "attached board to use"),
+ ),
+ server.ProjectOption(
+ "project_type",
+ help="Type of project to generate.",
+ choices=tuple(PROJECT_TYPES),
+ ),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption(
+ "west_cmd",
+ help=(
+ "Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable."
+ ),
+ ),
+ server.ProjectOption("zephyr_base", help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self, tvm_version):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path=""
+ if IS_TEMPLATE
+ else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS,
+ )
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def _create_prj_conf(self, project_dir, options):
+ with open(project_dir / "prj.conf", "w") as f:
+ f.write(
+ "# For UART used from main().\n"
+ "CONFIG_RING_BUFFER=y\n"
+ "CONFIG_UART_CONSOLE=n\n"
+ "CONFIG_UART_INTERRUPT_DRIVEN=y\n"
+ "\n"
+ )
+ f.write("# For TVMPlatformAbort().\n" "CONFIG_REBOOT=y\n" "\n")
+
+ if True: # options["project_type"] == "host_driven":
+ f.write("# For RPC server C++ bindings.\n" "CONFIG_CPLUSPLUS=y\n" "\n")
+
+ f.write("# For math routines\n" "CONFIG_NEWLIB_LIBC=y\n" "\n")
+
+ if self._has_fpu(options["zephyr_board"]):
+ f.write("# For models with floating point.\n" "CONFIG_FPU=y\n" "\n")
+
+ main_stack_size = None
+ if self._is_qemu(options) and options["project_type"] == "host_driven":
+ main_stack_size = 1536
+
+ # Set main stack size, if needed.
+ if main_stack_size is not None:
+ f.write(f"CONFIG_MAIN_STACK_SIZE={main_stack_size}\n")
+
+ f.write("# For random number generation.\n" "CONFIG_TEST_RANDOM_GENERATOR=y\n")
+
+ if self._is_qemu(options):
+ f.write("CONFIG_TIMER_RANDOM_GENERATOR=y\n")
+ else:
+ f.write("CONFIG_ENTROPY_GENERATOR=y\n")
+ f.write("\n")
+
+ API_SERVER_CRT_LIBS_TOKEN = "<API_SERVER_CRT_LIBS>"
+
+ CRT_LIBS_BY_PROJECT_TYPE = {
+ "host_driven": "microtvm_rpc_server microtvm_rpc_common common",
+ "aot_demo": "aot_executor memory microtvm_rpc_common common",
+ }
Review comment:
Is self-hosted dropped in favor of AOT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mehrdadh commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
mehrdadh commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r666431843
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,533 @@
+import collections
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = '' if 'cwd' not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.Mapping):
+
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options, cmake_entries):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}")
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(
+ f'No attached boards recognized by {" ".join(nrfjprog_args)}'
+ )
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options['nrfjprog_snr'] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: "
+ f'{", ".join(boards)}'
+ )
+
+ if str(options['nrfjprog_snr']) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options['nrfjprog_snr']]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption("gdbserver_port", help=("If given, port number to use when running the "
+ "local gdbserver")),
+ server.ProjectOption("openocd_serial", help=("When used with OpenOCD targets, serial # of the "
+ "attached board to use")),
+ server.ProjectOption("nrfjprog_snr", help=("When used with nRF targets, serial # of the "
+ "attached board to use, from nrfjprog")),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption("west_cmd",
+ help=("Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable.")),
+ server.ProjectOption("zephyr_base",
+ help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path="" if IS_TEMPLATE else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS)
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def generate_project(self, model_library_format_path, standalone_crt_dir, project_dir, options):
+ project_dir = pathlib.Path(project_dir)
+ # Make project directory.
+ project_dir.mkdir()
+
+ # Copy ourselves to the generated project. TVM may perform further build steps on the generated project
+ # by launching the copy.
+ shutil.copy2(__file__, project_dir / os.path.basename(__file__))
+
+ # Place Model Library Format tarball in the special location, which this script uses to decide
+ # whether it's being invoked in a template or generated project.
+ project_model_library_format_tar_path = project_dir / MODEL_LIBRARY_FORMAT_RELPATH
+ shutil.copy2(model_library_format_path, project_model_library_format_tar_path)
+
+ shutil.copytree(API_SERVER_DIR / "boards", project_dir / "boards")
+
+ # Extract Model Library Format tarball.into <project_dir>/model.
+ extract_path = os.path.splitext(project_model_library_format_tar_path)[0]
+ with tarfile.TarFile(project_model_library_format_tar_path) as tf:
+ os.makedirs(extract_path)
+ tf.extractall(path=extract_path)
+
+ if self._is_qemu(options):
+ shutil.copytree(API_SERVER_DIR / "qemu-hack", project_dir / "qemu-hack")
+
+ # Populate CRT.
+ crt_path = project_dir / "crt"
+ crt_path.mkdir()
+ for item in self.CRT_COPY_ITEMS:
+ src_path = os.path.join(standalone_crt_dir, item)
+ dst_path = crt_path / item
+ if os.path.isdir(src_path):
+ shutil.copytree(src_path, dst_path)
+ else:
+ shutil.copy2(src_path, dst_path)
+
+ # Populate Makefile.
+ shutil.copy2(API_SERVER_DIR / "CMakeLists.txt", project_dir / "CMakeLists.txt")
+ shutil.copy2(API_SERVER_DIR / "prj.conf", project_dir / "prj.conf")
+
+ # Populate crt-config.h
+ crt_config_dir = project_dir / "crt_config"
+ crt_config_dir.mkdir()
+ shutil.copy2(API_SERVER_DIR / "crt_config" / "crt_config.h", crt_config_dir / "crt_config.h")
+
+ # Populate src/
+ src_dir = project_dir / "src"
+ src_dir.mkdir()
+ shutil.copy2(API_SERVER_DIR / "src" / "main.c", src_dir / "main.c")
+
+ def build(self, options):
+ BUILD_DIR.mkdir()
+
+ cmake_args = ["cmake", ".."]
+ if options.get("verbose"):
+ cmake_args.append("-DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE")
+
+ if options.get("zephyr_base"):
+ cmake_args.append(f"-DZEPHYR_BASE:STRING={options['zephyr_base']}")
+
+ cmake_args.append(f"-DBOARD:STRING={options['zephyr_board']}")
+
+ check_call(cmake_args, cwd=BUILD_DIR)
+
+ args = ["make", "-j2"]
+ if options.get("verbose"):
+ args.append("VERBOSE=1")
+ check_call(args, cwd=BUILD_DIR)
+
+ @classmethod
+ def _is_qemu(cls, options):
+ return "qemu" in options["zephyr_board"]
+
+ def flash(self, options):
+ if self._is_qemu(options):
+ return # NOTE: qemu requires no flash step--it is launched from open_transport.
+
+ zephyr_board = options["zephyr_board"]
+
+ # The nRF5340DK requires an additional `nrfjprog --recover` before each flash cycle.
+ # This is because readback protection is enabled by default when this device is flashed.
+ # Otherwise, flashing may fail with an error such as the following:
+ # ERROR: The operation attempted is unavailable due to readback protection in
+ # ERROR: your device. Please use --recover to unlock the device.
+ if zephyr_board.startswith("nrf5340dk") and _get_flash_runner() == "nrfjprog":
+ recover_args = ["nrfjprog", "--recover"]
+ recover_args.extend(_get_nrf_device_args(options))
+ self._subprocess_env.run(recover_args, cwd=build_dir)
Review comment:
`build_dir` is also undefined
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tmoreau89 commented on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
tmoreau89 commented on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-894692924
Thank you @areusch , @leandron, @gromero, @guberti, @mehrdadh the PR was merged!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r677838273
##########
File path: apps/bundle_deploy/crt_config/crt_config.h
##########
@@ -35,9 +35,9 @@
/*! Maximum supported arguments in generated functions */
#define TVM_CRT_MAX_ARGS 10
/*! Maximum supported string length in dltype, e.g. "int8", "int16", "float32" */
-#define TVM_CRT_STRLEN_DLTYPE 10
+#define TVM_CRT_MAX_STRLEN_DLTYPE 10
Review comment:
There is still a `crt/` in `template_project` besides `crt_config/`, I believe the former should be deleted?
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,673 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import collections
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.Mapping):
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}"
+ )
+
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_l4r5zi": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(f'No attached boards recognized by {" ".join(nrfjprog_args)}')
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options["nrfjprog_snr"] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: " f'{", ".join(boards)}'
+ )
+
+ if str(options["nrfjprog_snr"]) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options["nrfjprog_snr"]]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_TYPES = []
+if IS_TEMPLATE:
+ for d in (API_SERVER_DIR / "src").iterdir():
+ if d.is_dir():
+ PROJECT_TYPES.append(d.name)
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption(
+ "extra_files",
+ help="If given, during generate_project, uncompress the tarball at this path into the project dir",
+ ),
+ server.ProjectOption(
+ "gdbserver_port", help=("If given, port number to use when running the local gdbserver")
+ ),
+ server.ProjectOption(
+ "nrfjprog_snr",
+ help=(
+ "When used with nRF targets, serial # of the " "attached board to use, from nrfjprog"
+ ),
+ ),
+ server.ProjectOption(
+ "openocd_serial",
+ help=("When used with OpenOCD targets, serial # of the " "attached board to use"),
+ ),
+ server.ProjectOption(
+ "project_type",
+ help="Type of project to generate.",
+ choices=tuple(PROJECT_TYPES),
+ ),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption(
+ "west_cmd",
+ help=(
+ "Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable."
+ ),
+ ),
+ server.ProjectOption("zephyr_base", help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self, tvm_version):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path=""
+ if IS_TEMPLATE
+ else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS,
+ )
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def _create_prj_conf(self, project_dir, options):
+ with open(project_dir / "prj.conf", "w") as f:
+ f.write(
+ "# For UART used from main().\n"
+ "CONFIG_RING_BUFFER=y\n"
+ "CONFIG_UART_CONSOLE=n\n"
+ "CONFIG_UART_INTERRUPT_DRIVEN=y\n"
+ "\n"
+ )
+ f.write("# For TVMPlatformAbort().\n" "CONFIG_REBOOT=y\n" "\n")
+
+ if True: # options["project_type"] == "host_driven":
+ f.write("# For RPC server C++ bindings.\n" "CONFIG_CPLUSPLUS=y\n" "\n")
+
+ f.write("# For math routines\n" "CONFIG_NEWLIB_LIBC=y\n" "\n")
+
+ f.write("# For models with floating point.\n" "CONFIG_FPU=y\n" "\n")
+
+ main_stack_size = None
+ if self._is_qemu(options) and options["project_type"] == "host_driven":
+ main_stack_size = 1536
+
+ # Set main stack size, if needed.
+ if main_stack_size is not None:
+ f.write(f"CONFIG_MAIN_STACK_SIZE={main_stack_size}\n")
+
+ f.write("# For random number generation.\n" "CONFIG_TEST_RANDOM_GENERATOR=y\n")
+
+ if self._is_qemu(options):
+ f.write("CONFIG_TIMER_RANDOM_GENERATOR=y\n")
+ else:
+ f.write("CONFIG_ENTROPY_GENERATOR=y\n")
+ f.write("\n")
+
+ API_SERVER_CRT_LIBS_TOKEN = "<API_SERVER_CRT_LIBS>"
+
+ CRT_LIBS_BY_PROJECT_TYPE = {
+ "host_driven": "microtvm_rpc_server microtvm_rpc_common common",
+ "aot_demo": "aot_executor memory microtvm_rpc_common common",
+ }
+
+ def generate_project(self, model_library_format_path, standalone_crt_dir, project_dir, options):
+ project_dir = pathlib.Path(project_dir)
+ # Make project directory.
+ project_dir.mkdir()
+
+ # Copy ourselves to the generated project. TVM may perform further build steps on the generated project
+ # by launching the copy.
+ shutil.copy2(__file__, project_dir / os.path.basename(__file__))
+
+ # Place Model Library Format tarball in the special location, which this script uses to decide
+ # whether it's being invoked in a template or generated project.
+ project_model_library_format_tar_path = project_dir / MODEL_LIBRARY_FORMAT_RELPATH
+ shutil.copy2(model_library_format_path, project_model_library_format_tar_path)
+
+ # Extract Model Library Format tarball.into <project_dir>/model.
+ extract_path = os.path.splitext(project_model_library_format_tar_path)[0]
+ with tarfile.TarFile(project_model_library_format_tar_path) as tf:
+ os.makedirs(extract_path)
+ tf.extractall(path=extract_path)
+
+ if self._is_qemu(options):
+ shutil.copytree(API_SERVER_DIR / "qemu-hack", project_dir / "qemu-hack")
+
+ # Populate CRT.
+ crt_path = project_dir / "crt"
+ crt_path.mkdir()
+ for item in self.CRT_COPY_ITEMS:
+ src_path = os.path.join(standalone_crt_dir, item)
+ dst_path = crt_path / item
+ if os.path.isdir(src_path):
+ shutil.copytree(src_path, dst_path)
+ else:
+ shutil.copy2(src_path, dst_path)
+
+ # Populate Makefile.
+ with open(API_SERVER_DIR / "CMakeLists.txt.template", "r") as cmake_template_f:
+ with open(project_dir / "CMakeLists.txt", "w") as cmake_f:
+ for line in cmake_template_f:
+ if self.API_SERVER_CRT_LIBS_TOKEN in line:
+ crt_libs = self.CRT_LIBS_BY_PROJECT_TYPE[options["project_type"]]
+ line = line.replace("<API_SERVER_CRT_LIBS>", crt_libs)
+
+ cmake_f.write(line)
+
+ self._create_prj_conf(project_dir, options)
+
+ # Populate crt-config.h
+ crt_config_dir = project_dir / "crt_config"
+ crt_config_dir.mkdir()
+ shutil.copy2(
+ API_SERVER_DIR / "crt_config" / "crt_config.h", crt_config_dir / "crt_config.h"
+ )
+
+ # Populate src/
+ src_dir = project_dir / "src"
+ shutil.copytree(API_SERVER_DIR / "src" / options["project_type"], src_dir)
+
+ # Populate extra_files
+ if options.get("extra_files_tar"):
+ with tarfile.open(options["extra_files_tar"], mode="r:*") as tf:
+ tf.extractall(project_dir)
+
+ def build(self, options):
+ BUILD_DIR.mkdir()
+
+ cmake_args = ["cmake", ".."]
+ if options.get("verbose"):
+ cmake_args.append("-DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE")
+
+ if options.get("zephyr_base"):
+ cmake_args.append(f"-DZEPHYR_BASE:STRING={options['zephyr_base']}")
+
+ cmake_args.append(f"-DBOARD:STRING={options['zephyr_board']}")
+
+ check_call(cmake_args, cwd=BUILD_DIR)
+
+ args = ["make", "-j2"]
+ if options.get("verbose"):
+ args.append("VERBOSE=1")
+ check_call(args, cwd=BUILD_DIR)
+
+ @classmethod
+ def _is_qemu(cls, options):
+ return (
+ "qemu" in options["zephyr_board"]
+ or
+ # NOTE: mps2_an521 naming breaks the rule that "qemu" is included when qemu is the
+ # launch method.
+ "mps2_an521" in options["zephyr_board"]
Review comment:
I think it's ok to add a list of boards here instead using the `-qemu` suffix trick as in the old API. However I just would like to mention that there are other boards on Zephyr in the same situation, like the new mps3 boards, and probably there will be more in the future. Zephyr doesn't have a policy to enforce the `qemu-` prefix afaik, unfortunately. That's why I introduced the `-qemu` suffix, so one could add more boards like `mps2_an521` to microTVM without having to update the list here and just using the `-qemu` suffix`, so it would be a bit more generic way to handle such a kind of boards.
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,673 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import collections
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.Mapping):
Review comment:
I think it could be rewritten as:
```
class CMakeCache(collections.abc.Mapping):
```
so adding `abc` without prejudice in order to avoid deprecation warnings?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r677865293
##########
File path: python/tvm/micro/project.py
##########
@@ -0,0 +1,97 @@
+"""Defines glue wrappers around the Project API which mate to TVM interfaces."""
+
+from ..contrib import utils
+from .build import get_standalone_crt_dir
+from .model_library_format import export_model_library_format
+from .project_api import client
+from .transport import Transport, TransportTimeouts
+
+
+class ProjectTransport(Transport):
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._timeouts = None
+
+ def timeouts(self):
+ assert self._timeouts is not None, "Transport not yet opened"
+ return self._timeouts
+
+ def open(self):
+ reply = self._client.connect_transport(self._options)
+ self._timeouts = TransportTimeouts(**reply["timeouts"])
+
+ def close(self):
+ self._client.disconnect_transport()
+
+ def write(self, data, timeout_sec):
+ return self._client.write_transport(data, timeout_sec)["bytes_written"]
+
+ def read(self, n, timeout_sec):
+ return self._client.read_transport(n, timeout_sec)["data"]
+
+
+class TemplateProjectError(Exception):
+ """Raised when the Project API server given to GeneratedProject reports is_template=True."""
+
+
+class GeneratedProject:
+ """Defines a glue interface to interact with a generated project through the API server."""
+
+ @classmethod
+ def from_directory(cls, project_dir, options):
+ return cls(client.instantiate_from_dir(project_dir), options)
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._info = self._client.server_info_query()
+ if self._info['is_template']:
+ raise TemplateProjectError()
+
Review comment:
@gromero ah gotcha. yeah so i view this more as an internal interface rather than an external one. as such, i do think that a server_info_query() may have a place here. maybe better for your use case would be a function `get_project_options` which would call `server_info_query` if it hadn't yet been, and then return the `project_options`. the idea here is to document all of TVM's uses of Project API all in once place. since there isn't that use case today in the code, it isn't there yet. however, you should feel free to add such functions to project.py as needed.
does that sound ok to you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r679553356
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,692 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import collections
+import collections.abc
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.abc.Mapping):
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}"
+ )
+
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_l4r5zi": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(f'No attached boards recognized by {" ".join(nrfjprog_args)}')
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options["nrfjprog_snr"] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: " f'{", ".join(boards)}'
+ )
+
+ if str(options["nrfjprog_snr"]) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options["nrfjprog_snr"]]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_TYPES = []
+if IS_TEMPLATE:
+ for d in (API_SERVER_DIR / "src").iterdir():
+ if d.is_dir():
+ PROJECT_TYPES.append(d.name)
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption(
+ "extra_files",
+ help="If given, during generate_project, uncompress the tarball at this path into the project dir",
+ ),
+ server.ProjectOption(
+ "gdbserver_port", help=("If given, port number to use when running the local gdbserver")
+ ),
+ server.ProjectOption(
+ "nrfjprog_snr",
+ help=(
+ "When used with nRF targets, serial # of the " "attached board to use, from nrfjprog"
+ ),
+ ),
+ server.ProjectOption(
+ "openocd_serial",
+ help=("When used with OpenOCD targets, serial # of the " "attached board to use"),
+ ),
+ server.ProjectOption(
+ "project_type",
+ help="Type of project to generate.",
+ choices=tuple(PROJECT_TYPES),
+ ),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption(
+ "west_cmd",
+ help=(
+ "Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable."
+ ),
+ ),
+ server.ProjectOption("zephyr_base", help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self, tvm_version):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path=""
+ if IS_TEMPLATE
+ else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS,
+ )
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def _create_prj_conf(self, project_dir, options):
+ with open(project_dir / "prj.conf", "w") as f:
+ f.write(
+ "# For UART used from main().\n"
+ "CONFIG_RING_BUFFER=y\n"
+ "CONFIG_UART_CONSOLE=n\n"
+ "CONFIG_UART_INTERRUPT_DRIVEN=y\n"
+ "\n"
+ )
+ f.write("# For TVMPlatformAbort().\n" "CONFIG_REBOOT=y\n" "\n")
+
+ if True: # options["project_type"] == "host_driven":
+ f.write("# For RPC server C++ bindings.\n" "CONFIG_CPLUSPLUS=y\n" "\n")
+
+ f.write("# For math routines\n" "CONFIG_NEWLIB_LIBC=y\n" "\n")
+
+ if self._has_fpu(options["zephyr_board"]):
+ f.write("# For models with floating point.\n" "CONFIG_FPU=y\n" "\n")
+
+ main_stack_size = None
+ if self._is_qemu(options) and options["project_type"] == "host_driven":
Review comment:
@mehrdadh I think CI won't catch the `tutorial/micro/micro_tflite.py`. On the other hand I'm ok fixing it on a follow-up PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tmoreau89 merged pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
tmoreau89 merged pull request #8380:
URL: https://github.com/apache/tvm/pull/8380
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mehrdadh commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
mehrdadh commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r666430797
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,533 @@
+import collections
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = '' if 'cwd' not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.Mapping):
+
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options, cmake_entries):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}")
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(
+ f'No attached boards recognized by {" ".join(nrfjprog_args)}'
+ )
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options['nrfjprog_snr'] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: "
+ f'{", ".join(boards)}'
+ )
+
+ if str(options['nrfjprog_snr']) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options['nrfjprog_snr']]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption("gdbserver_port", help=("If given, port number to use when running the "
+ "local gdbserver")),
+ server.ProjectOption("openocd_serial", help=("When used with OpenOCD targets, serial # of the "
+ "attached board to use")),
+ server.ProjectOption("nrfjprog_snr", help=("When used with nRF targets, serial # of the "
+ "attached board to use, from nrfjprog")),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption("west_cmd",
+ help=("Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable.")),
+ server.ProjectOption("zephyr_base",
+ help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path="" if IS_TEMPLATE else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS)
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def generate_project(self, model_library_format_path, standalone_crt_dir, project_dir, options):
+ project_dir = pathlib.Path(project_dir)
+ # Make project directory.
+ project_dir.mkdir()
+
+ # Copy ourselves to the generated project. TVM may perform further build steps on the generated project
+ # by launching the copy.
+ shutil.copy2(__file__, project_dir / os.path.basename(__file__))
+
+ # Place Model Library Format tarball in the special location, which this script uses to decide
+ # whether it's being invoked in a template or generated project.
+ project_model_library_format_tar_path = project_dir / MODEL_LIBRARY_FORMAT_RELPATH
+ shutil.copy2(model_library_format_path, project_model_library_format_tar_path)
+
+ shutil.copytree(API_SERVER_DIR / "boards", project_dir / "boards")
+
+ # Extract Model Library Format tarball.into <project_dir>/model.
+ extract_path = os.path.splitext(project_model_library_format_tar_path)[0]
+ with tarfile.TarFile(project_model_library_format_tar_path) as tf:
+ os.makedirs(extract_path)
+ tf.extractall(path=extract_path)
+
+ if self._is_qemu(options):
+ shutil.copytree(API_SERVER_DIR / "qemu-hack", project_dir / "qemu-hack")
+
+ # Populate CRT.
+ crt_path = project_dir / "crt"
+ crt_path.mkdir()
+ for item in self.CRT_COPY_ITEMS:
+ src_path = os.path.join(standalone_crt_dir, item)
+ dst_path = crt_path / item
+ if os.path.isdir(src_path):
+ shutil.copytree(src_path, dst_path)
+ else:
+ shutil.copy2(src_path, dst_path)
+
+ # Populate Makefile.
+ shutil.copy2(API_SERVER_DIR / "CMakeLists.txt", project_dir / "CMakeLists.txt")
+ shutil.copy2(API_SERVER_DIR / "prj.conf", project_dir / "prj.conf")
+
+ # Populate crt-config.h
+ crt_config_dir = project_dir / "crt_config"
+ crt_config_dir.mkdir()
+ shutil.copy2(API_SERVER_DIR / "crt_config" / "crt_config.h", crt_config_dir / "crt_config.h")
+
+ # Populate src/
+ src_dir = project_dir / "src"
+ src_dir.mkdir()
+ shutil.copy2(API_SERVER_DIR / "src" / "main.c", src_dir / "main.c")
+
+ def build(self, options):
+ BUILD_DIR.mkdir()
+
+ cmake_args = ["cmake", ".."]
+ if options.get("verbose"):
+ cmake_args.append("-DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE")
+
+ if options.get("zephyr_base"):
+ cmake_args.append(f"-DZEPHYR_BASE:STRING={options['zephyr_base']}")
+
+ cmake_args.append(f"-DBOARD:STRING={options['zephyr_board']}")
+
+ check_call(cmake_args, cwd=BUILD_DIR)
+
+ args = ["make", "-j2"]
+ if options.get("verbose"):
+ args.append("VERBOSE=1")
+ check_call(args, cwd=BUILD_DIR)
+
+ @classmethod
+ def _is_qemu(cls, options):
+ return "qemu" in options["zephyr_board"]
+
+ def flash(self, options):
+ if self._is_qemu(options):
+ return # NOTE: qemu requires no flash step--it is launched from open_transport.
+
+ zephyr_board = options["zephyr_board"]
+
+ # The nRF5340DK requires an additional `nrfjprog --recover` before each flash cycle.
+ # This is because readback protection is enabled by default when this device is flashed.
+ # Otherwise, flashing may fail with an error such as the following:
+ # ERROR: The operation attempted is unavailable due to readback protection in
+ # ERROR: your device. Please use --recover to unlock the device.
+ if zephyr_board.startswith("nrf5340dk") and _get_flash_runner() == "nrfjprog":
+ recover_args = ["nrfjprog", "--recover"]
+ recover_args.extend(_get_nrf_device_args(options))
+ self._subprocess_env.run(recover_args, cwd=build_dir)
Review comment:
`self._subprocess_env` is not defined. Also `SubprocessEnv` is missing in this file. Maybe copy is from zephyr.py to this file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-895298535
> I discussed with @areusch one small usability point that we well need to aadress later, which is basically that the template project won't be added to Python packages generated from the TVM repo _as is_.
>
> We'll need to offer some mechanism (and documentation) for a user to point to a Git repo and a path to access the template project to be used, or even maintaining an index of known-good repos and paths we know. This can be done via APIs or indeed via `tvmc` (cc @gromero).
>
> It is probably something that will confues people who are not familiar with the code base and just want to use TVM as a tool.
@leandron yeah, that's a good point. I was not aware that the template dirs would be out of the currently existing TVM Python packages. How is the code in 'apps' handled currently in general? Would it make sense have another package just for the templates and add it as a dependency when on installs the other TVM packages? Or even leave the package install as optional and `tvmc` would warn the user about it being absent (not installed) - current `tvmc micro` allows two ways to specify the template dirs: 1) by project type (passing `-t zephyr` or `-t arduino`, for instance), and 2) passing a `PATH` to a template dir. Option 1. is where TVMC can detect the missing server in the default template dirs and warn the user, saying the user can install the optional package or go for using option 2., in case the template dirs are in ad hoc location.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r673520171
##########
File path: python/tvm/micro/compiler.py
##########
@@ -1,361 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
so i was thinking about this, one thing is that i had to make breaking changes to the `apps/microtvm/zephyr` code. i don't mind leaving in the old `tvm.micro.Compiler` stuff (though I'd need to adapt more Transport layers), but it might be confusing to have two different styles of Zephyr template projects. we haven't done a TVM release with this code in there, so i wonder if it's necessary to keep the Compiler stuff in there. wdyt?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r677863154
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,673 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import collections
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.Mapping):
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}"
+ )
+
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_l4r5zi": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(f'No attached boards recognized by {" ".join(nrfjprog_args)}')
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options["nrfjprog_snr"] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: " f'{", ".join(boards)}'
+ )
+
+ if str(options["nrfjprog_snr"]) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options["nrfjprog_snr"]]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_TYPES = []
+if IS_TEMPLATE:
+ for d in (API_SERVER_DIR / "src").iterdir():
+ if d.is_dir():
+ PROJECT_TYPES.append(d.name)
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption(
+ "extra_files",
+ help="If given, during generate_project, uncompress the tarball at this path into the project dir",
+ ),
+ server.ProjectOption(
+ "gdbserver_port", help=("If given, port number to use when running the local gdbserver")
+ ),
+ server.ProjectOption(
+ "nrfjprog_snr",
+ help=(
+ "When used with nRF targets, serial # of the " "attached board to use, from nrfjprog"
+ ),
+ ),
+ server.ProjectOption(
+ "openocd_serial",
+ help=("When used with OpenOCD targets, serial # of the " "attached board to use"),
+ ),
+ server.ProjectOption(
+ "project_type",
+ help="Type of project to generate.",
+ choices=tuple(PROJECT_TYPES),
+ ),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption(
+ "west_cmd",
+ help=(
+ "Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable."
+ ),
+ ),
+ server.ProjectOption("zephyr_base", help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self, tvm_version):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path=""
+ if IS_TEMPLATE
+ else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS,
+ )
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def _create_prj_conf(self, project_dir, options):
+ with open(project_dir / "prj.conf", "w") as f:
+ f.write(
+ "# For UART used from main().\n"
+ "CONFIG_RING_BUFFER=y\n"
+ "CONFIG_UART_CONSOLE=n\n"
+ "CONFIG_UART_INTERRUPT_DRIVEN=y\n"
+ "\n"
+ )
+ f.write("# For TVMPlatformAbort().\n" "CONFIG_REBOOT=y\n" "\n")
+
+ if True: # options["project_type"] == "host_driven":
+ f.write("# For RPC server C++ bindings.\n" "CONFIG_CPLUSPLUS=y\n" "\n")
+
+ f.write("# For math routines\n" "CONFIG_NEWLIB_LIBC=y\n" "\n")
+
+ f.write("# For models with floating point.\n" "CONFIG_FPU=y\n" "\n")
+
+ main_stack_size = None
+ if self._is_qemu(options) and options["project_type"] == "host_driven":
+ main_stack_size = 1536
+
+ # Set main stack size, if needed.
+ if main_stack_size is not None:
+ f.write(f"CONFIG_MAIN_STACK_SIZE={main_stack_size}\n")
+
+ f.write("# For random number generation.\n" "CONFIG_TEST_RANDOM_GENERATOR=y\n")
+
+ if self._is_qemu(options):
+ f.write("CONFIG_TIMER_RANDOM_GENERATOR=y\n")
+ else:
+ f.write("CONFIG_ENTROPY_GENERATOR=y\n")
+ f.write("\n")
+
+ API_SERVER_CRT_LIBS_TOKEN = "<API_SERVER_CRT_LIBS>"
+
+ CRT_LIBS_BY_PROJECT_TYPE = {
+ "host_driven": "microtvm_rpc_server microtvm_rpc_common common",
+ "aot_demo": "aot_executor memory microtvm_rpc_common common",
+ }
+
+ def generate_project(self, model_library_format_path, standalone_crt_dir, project_dir, options):
+ project_dir = pathlib.Path(project_dir)
+ # Make project directory.
+ project_dir.mkdir()
+
+ # Copy ourselves to the generated project. TVM may perform further build steps on the generated project
+ # by launching the copy.
+ shutil.copy2(__file__, project_dir / os.path.basename(__file__))
+
+ # Place Model Library Format tarball in the special location, which this script uses to decide
+ # whether it's being invoked in a template or generated project.
+ project_model_library_format_tar_path = project_dir / MODEL_LIBRARY_FORMAT_RELPATH
+ shutil.copy2(model_library_format_path, project_model_library_format_tar_path)
+
+ # Extract Model Library Format tarball.into <project_dir>/model.
+ extract_path = os.path.splitext(project_model_library_format_tar_path)[0]
+ with tarfile.TarFile(project_model_library_format_tar_path) as tf:
+ os.makedirs(extract_path)
+ tf.extractall(path=extract_path)
+
+ if self._is_qemu(options):
+ shutil.copytree(API_SERVER_DIR / "qemu-hack", project_dir / "qemu-hack")
+
+ # Populate CRT.
+ crt_path = project_dir / "crt"
+ crt_path.mkdir()
+ for item in self.CRT_COPY_ITEMS:
+ src_path = os.path.join(standalone_crt_dir, item)
+ dst_path = crt_path / item
+ if os.path.isdir(src_path):
+ shutil.copytree(src_path, dst_path)
+ else:
+ shutil.copy2(src_path, dst_path)
+
+ # Populate Makefile.
+ with open(API_SERVER_DIR / "CMakeLists.txt.template", "r") as cmake_template_f:
+ with open(project_dir / "CMakeLists.txt", "w") as cmake_f:
+ for line in cmake_template_f:
+ if self.API_SERVER_CRT_LIBS_TOKEN in line:
+ crt_libs = self.CRT_LIBS_BY_PROJECT_TYPE[options["project_type"]]
+ line = line.replace("<API_SERVER_CRT_LIBS>", crt_libs)
+
+ cmake_f.write(line)
+
+ self._create_prj_conf(project_dir, options)
+
+ # Populate crt-config.h
+ crt_config_dir = project_dir / "crt_config"
+ crt_config_dir.mkdir()
+ shutil.copy2(
+ API_SERVER_DIR / "crt_config" / "crt_config.h", crt_config_dir / "crt_config.h"
+ )
+
+ # Populate src/
+ src_dir = project_dir / "src"
+ shutil.copytree(API_SERVER_DIR / "src" / options["project_type"], src_dir)
+
+ # Populate extra_files
+ if options.get("extra_files_tar"):
+ with tarfile.open(options["extra_files_tar"], mode="r:*") as tf:
+ tf.extractall(project_dir)
+
+ def build(self, options):
+ BUILD_DIR.mkdir()
+
+ cmake_args = ["cmake", ".."]
+ if options.get("verbose"):
+ cmake_args.append("-DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE")
+
+ if options.get("zephyr_base"):
+ cmake_args.append(f"-DZEPHYR_BASE:STRING={options['zephyr_base']}")
+
+ cmake_args.append(f"-DBOARD:STRING={options['zephyr_board']}")
+
+ check_call(cmake_args, cwd=BUILD_DIR)
+
+ args = ["make", "-j2"]
+ if options.get("verbose"):
+ args.append("VERBOSE=1")
+ check_call(args, cwd=BUILD_DIR)
+
+ @classmethod
+ def _is_qemu(cls, options):
+ return (
+ "qemu" in options["zephyr_board"]
+ or
+ # NOTE: mps2_an521 naming breaks the rule that "qemu" is included when qemu is the
+ # launch method.
+ "mps2_an521" in options["zephyr_board"]
Review comment:
ah i see. the main reason i went away from it was that the parameter was named `zephyr_board`, so a new user would be confused why they need to add `-qemu` suffix. happy to entertain other parameter names if you have suggestions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r677884779
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,673 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import collections
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.Mapping):
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tmoreau89 commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
tmoreau89 commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r683865885
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,698 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import atexit
+import collections
+import collections.abc
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.abc.Mapping):
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}"
+ )
+
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_l4r5zi": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(f'No attached boards recognized by {" ".join(nrfjprog_args)}')
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options["nrfjprog_snr"] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: " f'{", ".join(boards)}'
+ )
+
+ if str(options["nrfjprog_snr"]) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options["nrfjprog_snr"]]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_TYPES = []
+if IS_TEMPLATE:
+ for d in (API_SERVER_DIR / "src").iterdir():
+ if d.is_dir():
+ PROJECT_TYPES.append(d.name)
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption(
+ "extra_files",
+ help="If given, during generate_project, uncompress the tarball at this path into the project dir",
+ ),
+ server.ProjectOption(
+ "gdbserver_port", help=("If given, port number to use when running the local gdbserver")
+ ),
+ server.ProjectOption(
+ "nrfjprog_snr",
+ help=(
+ "When used with nRF targets, serial # of the " "attached board to use, from nrfjprog"
+ ),
+ ),
+ server.ProjectOption(
+ "openocd_serial",
+ help=("When used with OpenOCD targets, serial # of the " "attached board to use"),
+ ),
+ server.ProjectOption(
+ "project_type",
+ help="Type of project to generate.",
+ choices=tuple(PROJECT_TYPES),
+ ),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption(
+ "west_cmd",
+ help=(
+ "Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable."
+ ),
+ ),
+ server.ProjectOption("zephyr_base", help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self, tvm_version):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path=""
+ if IS_TEMPLATE
+ else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS,
+ )
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def _create_prj_conf(self, project_dir, options):
+ with open(project_dir / "prj.conf", "w") as f:
+ f.write(
+ "# For UART used from main().\n"
+ "CONFIG_RING_BUFFER=y\n"
+ "CONFIG_UART_CONSOLE=n\n"
+ "CONFIG_UART_INTERRUPT_DRIVEN=y\n"
+ "\n"
+ )
+ f.write("# For TVMPlatformAbort().\n" "CONFIG_REBOOT=y\n" "\n")
+
+ if True: # options["project_type"] == "host_driven":
Review comment:
you may want to clean up that comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r673519098
##########
File path: python/tvm/micro/project.py
##########
@@ -0,0 +1,97 @@
+"""Defines glue wrappers around the Project API which mate to TVM interfaces."""
+
+from ..contrib import utils
+from .build import get_standalone_crt_dir
+from .model_library_format import export_model_library_format
+from .project_api import client
+from .transport import Transport, TransportTimeouts
+
+
+class ProjectTransport(Transport):
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._timeouts = None
+
+ def timeouts(self):
+ assert self._timeouts is not None, "Transport not yet opened"
+ return self._timeouts
+
+ def open(self):
+ reply = self._client.connect_transport(self._options)
+ self._timeouts = TransportTimeouts(**reply["timeouts"])
+
+ def close(self):
+ self._client.disconnect_transport()
+
+ def write(self, data, timeout_sec):
+ return self._client.write_transport(data, timeout_sec)["bytes_written"]
+
+ def read(self, n, timeout_sec):
+ return self._client.read_transport(n, timeout_sec)["data"]
+
+
+class TemplateProjectError(Exception):
+ """Raised when the Project API server given to GeneratedProject reports is_template=True."""
+
+
+class GeneratedProject:
+ """Defines a glue interface to interact with a generated project through the API server."""
+
+ @classmethod
+ def from_directory(cls, project_dir, options):
+ return cls(client.instantiate_from_dir(project_dir), options)
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._info = self._client.server_info_query()
+ if self._info['is_template']:
+ raise TemplateProjectError()
+
Review comment:
could you say why? right now this code is just attempting to document the uses of project_api. i think we don't need to expose that here if it's not called. wdyt?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero edited a comment on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero edited a comment on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-888658001
@areusch Fixes in https://github.com/apache/tvm/pull/8380/commits/0d0c504a158ac9e03c301b88a5ca1127c57bd06e are good. Thanks. I think the only remaining issue is that `prj.conf` generation mechanism currently doesn't account for boards that don't have a FPU (like the `mps2_an521` board), so `CONFIG_FPY=y ` is set for every board. So I think we need to fix it because it's a regression of https://github.com/apache/tvm/pull/8055. Otherwise I think it's pretty much ready to land once CI is happy too :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-895336950
@gromero right now we haven't broadly addressed data dependencies in the wheel. i think we need to at some point build a central e.g. `tvm.utils.get_packaged_resource("...")` function to mimic `tvm.micro.get_standalone_crt_dir` (and use the former internally in the latter). this will allow us to add arbitrary data dependencies to the wheel and refer to their paths later on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero edited a comment on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero edited a comment on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-895298535
> I discussed with @areusch one small usability point that we well need to aadress later, which is basically that the template project won't be added to Python packages generated from the TVM repo _as is_.
>
> We'll need to offer some mechanism (and documentation) for a user to point to a Git repo and a path to access the template project to be used, or even maintaining an index of known-good repos and paths we know. This can be done via APIs or indeed via `tvmc` (cc @gromero).
>
> It is probably something that will confues people who are not familiar with the code base and just want to use TVM as a tool.
@leandron yeah, that's a good point. I was not aware that the template dirs would be out of the current existing TVM Python packages. How is the code in 'apps' handled currently in general? Would it make sense have another package just for the templates and add it as a dependency when on installs the other TVM packages? Or even leave the package install as optional and `tvmc` would warn the user about it being absent (not installed) - current `tvmc micro` allows two ways to specify the template dirs: 1) by project type (passing `-t zephyr` or `-t arduino`, for instance), and 2) passing a `PATH` to a template dir. Option 1. is where TVMC can detect the missing server in the default template dirs and warn the user, saying the user can install the optional package or go for using option 2., in case the template dirs are in ad hoc location.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tmoreau89 commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
tmoreau89 commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r683865774
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,698 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import atexit
+import collections
+import collections.abc
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.abc.Mapping):
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}"
+ )
+
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
Review comment:
Does the `nrf5340dk_nrf5340_cpuapp` need to be in there?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mehrdadh commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
mehrdadh commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r666578080
##########
File path: python/tvm/micro/compiler.py
##########
@@ -1,361 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
maybe keep this and add warning that it will be removed in the next version?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mehrdadh commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
mehrdadh commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r679546199
##########
File path: apps/microtvm/zephyr/template_project/microtvm_api_server.py
##########
@@ -0,0 +1,692 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import collections
+import collections.abc
+import enum
+import fcntl
+import logging
+import os
+import os.path
+import pathlib
+import queue
+import re
+import select
+import shlex
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+
+import serial
+import serial.tools.list_ports
+import yaml
+
+from tvm.micro.project_api import server
+
+
+_LOG = logging.getLogger(__name__)
+
+
+API_SERVER_DIR = pathlib.Path(os.path.dirname(__file__) or os.path.getcwd())
+
+
+BUILD_DIR = API_SERVER_DIR / "build"
+
+
+MODEL_LIBRARY_FORMAT_RELPATH = "model.tar"
+
+
+IS_TEMPLATE = not (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH).exists()
+
+
+def check_call(cmd_args, *args, **kwargs):
+ cwd_str = "" if "cwd" not in kwargs else f" (in cwd: {kwargs['cwd']})"
+ _LOG.info("run%s: %s", cwd_str, " ".join(shlex.quote(a) for a in cmd_args))
+ return subprocess.check_call(cmd_args, *args, **kwargs)
+
+
+CACHE_ENTRY_RE = re.compile(r"(?P<name>[^:]+):(?P<type>[^=]+)=(?P<value>.*)")
+
+
+CMAKE_BOOL_MAP = dict(
+ [(k, True) for k in ("1", "ON", "YES", "TRUE", "Y")]
+ + [(k, False) for k in ("0", "OFF", "NO", "FALSE", "N", "IGNORE", "NOTFOUND", "")]
+)
+
+
+class CMakeCache(collections.abc.Mapping):
+ def __init__(self, path):
+ self._path = path
+ self._dict = None
+
+ def __iter__(self):
+ return iter(self._dict)
+
+ def __getitem__(self, key):
+ if self._dict is None:
+ self._dict = self._read_cmake_cache()
+
+ return self._dict[key]
+
+ def __len__(self):
+ return len(self._dict)
+
+ def _read_cmake_cache(self):
+ """Read a CMakeCache.txt-like file and return a dictionary of values."""
+ entries = collections.OrderedDict()
+ with open(self._path, encoding="utf-8") as f:
+ for line in f:
+ m = CACHE_ENTRY_RE.match(line.rstrip("\n"))
+ if not m:
+ continue
+
+ if m.group("type") == "BOOL":
+ value = CMAKE_BOOL_MAP[m.group("value").upper()]
+ else:
+ value = m.group("value")
+
+ entries[m.group("name")] = value
+
+ return entries
+
+
+CMAKE_CACHE = CMakeCache(BUILD_DIR / "CMakeCache.txt")
+
+
+class BoardError(Exception):
+ """Raised when an attached board cannot be opened (i.e. missing /dev nodes, etc)."""
+
+
+class BoardAutodetectFailed(Exception):
+ """Raised when no attached hardware is found matching the board= given to ZephyrCompiler."""
+
+
+def _get_flash_runner():
+ flash_runner = CMAKE_CACHE.get("ZEPHYR_BOARD_FLASH_RUNNER")
+ if flash_runner is not None:
+ return flash_runner
+
+ with open(CMAKE_CACHE["ZEPHYR_RUNNERS_YAML"]) as f:
+ doc = yaml.load(f, Loader=yaml.FullLoader)
+ return doc["flash-runner"]
+
+
+def _get_device_args(options):
+ flash_runner = _get_flash_runner()
+
+ if flash_runner == "nrfjprog":
+ return _get_nrf_device_args(options)
+
+ if flash_runner == "openocd":
+ return _get_openocd_device_args(options)
+
+ raise BoardError(
+ f"Don't know how to find serial terminal for board {CMAKE_CACHE['BOARD']} with flash "
+ f"runner {flash_runner}"
+ )
+
+
+# kwargs passed to usb.core.find to find attached boards for the openocd flash runner.
+BOARD_USB_FIND_KW = {
+ "nucleo_l4r5zi": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "nucleo_f746zg": {"idVendor": 0x0483, "idProduct": 0x374B},
+ "stm32f746g_disco": {"idVendor": 0x0483, "idProduct": 0x374B},
+}
+
+
+def openocd_serial(options):
+ """Find the serial port to use for a board with OpenOCD flash strategy."""
+ if "openocd_serial" in options:
+ return options["openocd_serial"]
+
+ import usb # pylint: disable=import-outside-toplevel
+
+ find_kw = BOARD_USB_FIND_KW[CMAKE_CACHE["BOARD"]]
+ boards = usb.core.find(find_all=True, **find_kw)
+ serials = []
+ for b in boards:
+ serials.append(b.serial_number)
+
+ if len(serials) == 0:
+ raise BoardAutodetectFailed(f"No attached USB devices matching: {find_kw!r}")
+ serials.sort()
+
+ autodetected_openocd_serial = serials[0]
+ _LOG.debug("zephyr openocd driver: autodetected serial %s", serials[0])
+
+ return autodetected_openocd_serial
+
+
+def _get_openocd_device_args(options):
+ return ["--serial", openocd_serial(options)]
+
+
+def _get_nrf_device_args(options):
+ nrfjprog_args = ["nrfjprog", "--ids"]
+ nrfjprog_ids = subprocess.check_output(nrfjprog_args, encoding="utf-8")
+ if not nrfjprog_ids.strip("\n"):
+ raise BoardAutodetectFailed(f'No attached boards recognized by {" ".join(nrfjprog_args)}')
+
+ boards = nrfjprog_ids.split("\n")[:-1]
+ if len(boards) > 1:
+ if options["nrfjprog_snr"] is None:
+ raise BoardError(
+ "Multiple boards connected; specify one with nrfjprog_snr=: " f'{", ".join(boards)}'
+ )
+
+ if str(options["nrfjprog_snr"]) not in boards:
+ raise BoardError(
+ f"nrfjprog_snr ({options['nrfjprog_snr']}) not found in {nrfjprog_args}: {boards}"
+ )
+
+ return ["--snr", options["nrfjprog_snr"]]
+
+ if not boards:
+ return []
+
+ return ["--snr", boards[0]]
+
+
+PROJECT_TYPES = []
+if IS_TEMPLATE:
+ for d in (API_SERVER_DIR / "src").iterdir():
+ if d.is_dir():
+ PROJECT_TYPES.append(d.name)
+
+
+PROJECT_OPTIONS = [
+ server.ProjectOption(
+ "extra_files",
+ help="If given, during generate_project, uncompress the tarball at this path into the project dir",
+ ),
+ server.ProjectOption(
+ "gdbserver_port", help=("If given, port number to use when running the local gdbserver")
+ ),
+ server.ProjectOption(
+ "nrfjprog_snr",
+ help=(
+ "When used with nRF targets, serial # of the " "attached board to use, from nrfjprog"
+ ),
+ ),
+ server.ProjectOption(
+ "openocd_serial",
+ help=("When used with OpenOCD targets, serial # of the " "attached board to use"),
+ ),
+ server.ProjectOption(
+ "project_type",
+ help="Type of project to generate.",
+ choices=tuple(PROJECT_TYPES),
+ ),
+ server.ProjectOption("verbose", help="Run build with verbose output"),
+ server.ProjectOption(
+ "west_cmd",
+ help=(
+ "Path to the west tool. If given, supersedes both the zephyr_base "
+ "option and ZEPHYR_BASE environment variable."
+ ),
+ ),
+ server.ProjectOption("zephyr_base", help="Path to the zephyr base directory."),
+ server.ProjectOption("zephyr_board", help="Name of the Zephyr board to build for"),
+]
+
+
+class Handler(server.ProjectAPIHandler):
+ def __init__(self):
+ super(Handler, self).__init__()
+ self._proc = None
+
+ def server_info_query(self, tvm_version):
+ return server.ServerInfo(
+ platform_name="zephyr",
+ is_template=IS_TEMPLATE,
+ model_library_format_path=""
+ if IS_TEMPLATE
+ else (API_SERVER_DIR / MODEL_LIBRARY_FORMAT_RELPATH),
+ project_options=PROJECT_OPTIONS,
+ )
+
+ # These files and directories will be recursively copied into generated projects from the CRT.
+ CRT_COPY_ITEMS = ("include", "Makefile", "src")
+
+ def _create_prj_conf(self, project_dir, options):
+ with open(project_dir / "prj.conf", "w") as f:
+ f.write(
+ "# For UART used from main().\n"
+ "CONFIG_RING_BUFFER=y\n"
+ "CONFIG_UART_CONSOLE=n\n"
+ "CONFIG_UART_INTERRUPT_DRIVEN=y\n"
+ "\n"
+ )
+ f.write("# For TVMPlatformAbort().\n" "CONFIG_REBOOT=y\n" "\n")
+
+ if True: # options["project_type"] == "host_driven":
+ f.write("# For RPC server C++ bindings.\n" "CONFIG_CPLUSPLUS=y\n" "\n")
+
+ f.write("# For math routines\n" "CONFIG_NEWLIB_LIBC=y\n" "\n")
+
+ if self._has_fpu(options["zephyr_board"]):
+ f.write("# For models with floating point.\n" "CONFIG_FPU=y\n" "\n")
+
+ main_stack_size = None
+ if self._is_qemu(options) and options["project_type"] == "host_driven":
Review comment:
`aot_demo` condition is missing?
I think for that the stack size should be 2000
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero commented on a change in pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero commented on a change in pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#discussion_r678664098
##########
File path: python/tvm/micro/project.py
##########
@@ -0,0 +1,97 @@
+"""Defines glue wrappers around the Project API which mate to TVM interfaces."""
+
+from ..contrib import utils
+from .build import get_standalone_crt_dir
+from .model_library_format import export_model_library_format
+from .project_api import client
+from .transport import Transport, TransportTimeouts
+
+
+class ProjectTransport(Transport):
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._timeouts = None
+
+ def timeouts(self):
+ assert self._timeouts is not None, "Transport not yet opened"
+ return self._timeouts
+
+ def open(self):
+ reply = self._client.connect_transport(self._options)
+ self._timeouts = TransportTimeouts(**reply["timeouts"])
+
+ def close(self):
+ self._client.disconnect_transport()
+
+ def write(self, data, timeout_sec):
+ return self._client.write_transport(data, timeout_sec)["bytes_written"]
+
+ def read(self, n, timeout_sec):
+ return self._client.read_transport(n, timeout_sec)["data"]
+
+
+class TemplateProjectError(Exception):
+ """Raised when the Project API server given to GeneratedProject reports is_template=True."""
+
+
+class GeneratedProject:
+ """Defines a glue interface to interact with a generated project through the API server."""
+
+ @classmethod
+ def from_directory(cls, project_dir, options):
+ return cls(client.instantiate_from_dir(project_dir), options)
+
+ def __init__(self, client, options):
+ self._client = client
+ self._options = options
+ self._info = self._client.server_info_query()
+ if self._info['is_template']:
+ raise TemplateProjectError()
+
Review comment:
@areusch Sounds good!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
areusch commented on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-892996827
tested offline against nrf5340 and stm32f746xx_disco; waiting for CI to be fixed to retrigger. @gromero @mehrdadh can you take a look when you have a minute?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero edited a comment on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero edited a comment on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-888658001
@areusch Fixes in https://github.com/apache/tvm/pull/8380/commits/0d0c504a158ac9e03c301b88a5ca1127c57bd06e are good. Thanks. I think the only remaining issue is that `prj.conf` generation mechanism currently doesn't account for boards that don't have a FPU, so `CONFIG_FPU=y` (like the `mps2_an521` board). So I think we need to fix it because it's a regression of https://github.com/apache/tvm/pull/8055. Otherwise I think it's pretty much ready to land once CI is happy too :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gromero edited a comment on pull request #8380: [microTVM] Project API infrastructure
Posted by GitBox <gi...@apache.org>.
gromero edited a comment on pull request #8380:
URL: https://github.com/apache/tvm/pull/8380#issuecomment-888658001
@areusch Fixes in https://github.com/apache/tvm/pull/8380/commits/0d0c504a158ac9e03c301b88a5ca1127c57bd06e are good. Thanks. I think the only remaining issue is that `prj.conf` generation mechanism currently doesn't account for boards that don't have a FPU (like the `mps2_an521` board), so `CONFIG_FPY=y ` is set every board. So I think we need to fix it because it's a regression of https://github.com/apache/tvm/pull/8055. Otherwise I think it's pretty much ready to land once CI is happy too :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org