You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/09/18 21:58:47 UTC
[GitHub] [tvm] junrushao1994 opened a new pull request #9044: [Meta Schedule][M3b] Builder
junrushao1994 opened a new pull request #9044:
URL: https://github.com/apache/tvm/pull/9044
This PR is part of the stage M3b of the meta schedule project (https://github.com/apache/tvm/issues/8473).
This part is originally developed by Junru, Wuwei, Ruihang, Bohan, Siyuan, Hongyi, and later refactored by Junru and Xiyou with a new design and comprehensive unittests.
Review request: @jcf94 @comaniac @icemelon @zhiics @mbrookhart.
Co-authored-by: Xiyou Zhou \<xi...@octoml.ai>
Co-authored-by: Bohan Hou \<32...@users.noreply.github.com>
Co-authored-by: Ruihang Lai \<la...@qq.com>
Co-authored-by: Hongyi Jin \<32...@qq.com>
Co-authored-by: Wuwei Lin \<wu...@apache.org>
Co-authored-by: Siyuan Feng \<Hz...@sjtu.edu.cn>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac merged pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
comaniac merged pull request #9044:
URL: https://github.com/apache/tvm/pull/9044
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
zxybazh commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r712514948
##########
File path: python/tvm/relay/transform/transform.py
##########
@@ -1204,7 +1204,7 @@ def FakeQuantizationToInteger():
x w
| |
dq dq
- \ /
+ \\ /
Review comment:
Same here when I run the pytest, got a warning for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711697560
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
+ initializer: Optional[Callable[[], None]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ max_workers : Optional[int]
+ The maximum number of worker processes to be used.
+ Defaults to number of CPUs.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+ initializer : Optional[Callable[[], None]]
+ The initializer to be used for the worker processes.
+ """
+ super().__init__()
+
+ if max_workers is None:
+ max_workers = cpu_count()
Review comment:
This is good point. I imagine the main process will be stalled and waiting for the workers, so is it fine to use `cpu_count`? Let me know what you think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711708391
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
+ initializer: Optional[Callable[[], None]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ max_workers : Optional[int]
+ The maximum number of worker processes to be used.
+ Defaults to number of CPUs.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+ initializer : Optional[Callable[[], None]]
+ The initializer to be used for the worker processes.
+ """
+ super().__init__()
+
+ if max_workers is None:
+ max_workers = cpu_count()
+
+ self.pool = PopenPoolExecutor(
+ max_workers=max_workers,
+ timeout=timeout_sec,
+ initializer=initializer,
+ )
+ self.timeout_sec = timeout_sec
+ self.f_build = f_build
+ self.f_export = f_export
+ self._sanity_check()
+
+ def build(self, build_inputs: List[BuilderInput]) -> List[BuilderResult]:
+ results: List[BuilderResult] = []
+ map_result: MapResult
+
+ # Dispatch the build inputs to the worker processes.
+ for map_result in self.pool.map_with_error_catching(
+ lambda x: LocalBuilder._worker_func(*x),
+ [
+ (
+ self.f_build,
+ self.f_export,
+ build_input.mod,
+ build_input.target,
+ )
+ for build_input in build_inputs
+ ],
+ ):
+ if map_result.status == StatusKind.COMPLETE:
+ results.append(BuilderResult(map_result.value, None))
+ elif map_result.status == StatusKind.TIMEOUT:
+ results.append(
+ BuilderResult(
+ None,
+ f"LocalBuilder: Timeout, killed after {self.timeout_sec} seconds",
+ )
+ )
+ elif map_result.status == StatusKind.EXCEPTION:
+ results.append(
+ BuilderResult(
+ None,
+ "LocalBuilder: An exception occurred\n" + str(map_result.value),
+ )
+ )
+ else:
+ raise NotImplementedError(map_result.status)
Review comment:
Got it. Just updated the PR with a more detailed description indicating that the branch is unreachable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711692271
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
+ initializer: Optional[Callable[[], None]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ max_workers : Optional[int]
+ The maximum number of worker processes to be used.
+ Defaults to number of CPUs.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+ initializer : Optional[Callable[[], None]]
+ The initializer to be used for the worker processes.
+ """
+ super().__init__()
+
+ if max_workers is None:
+ max_workers = cpu_count()
Review comment:
nit: just my personal thought. EmpIrically, it might be better to use `cpu_count() - 1`.
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
+ initializer: Optional[Callable[[], None]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ max_workers : Optional[int]
+ The maximum number of worker processes to be used.
+ Defaults to number of CPUs.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+ initializer : Optional[Callable[[], None]]
+ The initializer to be used for the worker processes.
+ """
+ super().__init__()
+
+ if max_workers is None:
+ max_workers = cpu_count()
+
+ self.pool = PopenPoolExecutor(
+ max_workers=max_workers,
+ timeout=timeout_sec,
+ initializer=initializer,
+ )
+ self.timeout_sec = timeout_sec
+ self.f_build = f_build
+ self.f_export = f_export
+ self._sanity_check()
+
+ def build(self, build_inputs: List[BuilderInput]) -> List[BuilderResult]:
+ results: List[BuilderResult] = []
+ map_result: MapResult
+
+ # Dispatch the build inputs to the worker processes.
+ for map_result in self.pool.map_with_error_catching(
+ lambda x: LocalBuilder._worker_func(*x),
+ [
+ (
+ self.f_build,
+ self.f_export,
+ build_input.mod,
+ build_input.target,
+ )
+ for build_input in build_inputs
+ ],
+ ):
+ if map_result.status == StatusKind.COMPLETE:
+ results.append(BuilderResult(map_result.value, None))
+ elif map_result.status == StatusKind.TIMEOUT:
+ results.append(
+ BuilderResult(
+ None,
+ f"LocalBuilder: Timeout, killed after {self.timeout_sec} seconds",
+ )
+ )
+ elif map_result.status == StatusKind.EXCEPTION:
+ results.append(
+ BuilderResult(
+ None,
+ "LocalBuilder: An exception occurred\n" + str(map_result.value),
+ )
+ )
+ else:
+ raise NotImplementedError(map_result.status)
+ return results
+
+ def _sanity_check(self) -> None:
+ def _check(
+ f_build: Optional[str],
+ f_export: Optional[str],
+ ) -> None:
+ get_global_func_with_default_on_worker(name=f_build, default=None)
+ get_global_func_with_default_on_worker(name=f_export, default=None)
+
+ value = self.pool.submit(_check, self.f_build, self.f_export)
+ value.result()
+
+ @staticmethod
+ def _worker_func(
+ _f_build: Optional[str],
+ _f_export: Optional[str],
+ mod: IRModule,
+ target: Target,
+ ) -> str:
+ # Step 0. Get the registered functions
+ f_build: Callable[
+ [IRModule, Target],
+ Module,
+ ] = get_global_func_with_default_on_worker(_f_build, default_build)
+ f_export: Callable[
+ [Module],
+ str,
+ ] = get_global_func_with_default_on_worker(_f_export, default_export)
Review comment:
nit: You could use custom type such as `FBuildType = Callable[[IRModule, Target], Module,]` to avoid redundant complex type strings.
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
+ initializer: Optional[Callable[[], None]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ max_workers : Optional[int]
+ The maximum number of worker processes to be used.
+ Defaults to number of CPUs.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+ initializer : Optional[Callable[[], None]]
+ The initializer to be used for the worker processes.
+ """
+ super().__init__()
+
+ if max_workers is None:
+ max_workers = cpu_count()
+
+ self.pool = PopenPoolExecutor(
+ max_workers=max_workers,
+ timeout=timeout_sec,
+ initializer=initializer,
+ )
+ self.timeout_sec = timeout_sec
+ self.f_build = f_build
+ self.f_export = f_export
+ self._sanity_check()
+
+ def build(self, build_inputs: List[BuilderInput]) -> List[BuilderResult]:
+ results: List[BuilderResult] = []
+ map_result: MapResult
+
+ # Dispatch the build inputs to the worker processes.
+ for map_result in self.pool.map_with_error_catching(
+ lambda x: LocalBuilder._worker_func(*x),
+ [
+ (
+ self.f_build,
+ self.f_export,
+ build_input.mod,
+ build_input.target,
+ )
+ for build_input in build_inputs
+ ],
+ ):
+ if map_result.status == StatusKind.COMPLETE:
+ results.append(BuilderResult(map_result.value, None))
+ elif map_result.status == StatusKind.TIMEOUT:
+ results.append(
+ BuilderResult(
+ None,
+ f"LocalBuilder: Timeout, killed after {self.timeout_sec} seconds",
+ )
+ )
+ elif map_result.status == StatusKind.EXCEPTION:
+ results.append(
+ BuilderResult(
+ None,
+ "LocalBuilder: An exception occurred\n" + str(map_result.value),
+ )
+ )
+ else:
+ raise NotImplementedError(map_result.status)
Review comment:
Should we raise an exception or just return the not implemented status to keep it going?
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
Review comment:
Inconsistent typing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711707800
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
+ initializer: Optional[Callable[[], None]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ max_workers : Optional[int]
+ The maximum number of worker processes to be used.
+ Defaults to number of CPUs.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+ initializer : Optional[Callable[[], None]]
+ The initializer to be used for the worker processes.
+ """
+ super().__init__()
+
+ if max_workers is None:
+ max_workers = cpu_count()
Review comment:
Got it. AutoTVM/Ansor used to rely on python's native multiprocessing library, which involves good chunk of copy when creating processes. Even though fork implements CoW, when mutating anything, it still involves good chunk of memory.
In our new infra, we completely use popen pool, which creates completely separate processes as workers which could possibly reduce some memory consumption, because it might avoid unnecessary copy of anything.
On the other hand, in case that memory runs out, users could have the liberty to specify max_workers by hand, so I wouldn't worry too much :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711707800
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
+ initializer: Optional[Callable[[], None]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ max_workers : Optional[int]
+ The maximum number of worker processes to be used.
+ Defaults to number of CPUs.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+ initializer : Optional[Callable[[], None]]
+ The initializer to be used for the worker processes.
+ """
+ super().__init__()
+
+ if max_workers is None:
+ max_workers = cpu_count()
Review comment:
Got it. AutoTVM/Ansor used to rely on python's native multiprocessing library, which involves unnecessary copy of some global states when creating processes. Even though fork implements CoW, when mutating anything, it still involves good chunk of memory.
In our new infra, we completely use popen pool, which creates completely separate processes as workers which could possibly reduce some memory consumption, because it might avoid unnecessary copy of anything.
On the other hand, in case that memory runs out, users could have the liberty to specify max_workers by hand, so I wouldn't worry too much :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r712484171
##########
File path: python/tvm/relay/transform/transform.py
##########
@@ -1204,7 +1204,7 @@ def FakeQuantizationToInteger():
x w
| |
dq dq
- \ /
+ \\ /
Review comment:
Haha yeah it is intentional. Otherwise the unhappy pytest reports a warning demanding escape characters
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711706892
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
+ initializer: Optional[Callable[[], None]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ max_workers : Optional[int]
+ The maximum number of worker processes to be used.
+ Defaults to number of CPUs.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+ initializer : Optional[Callable[[], None]]
+ The initializer to be used for the worker processes.
+ """
+ super().__init__()
+
+ if max_workers is None:
+ max_workers = cpu_count()
+
+ self.pool = PopenPoolExecutor(
+ max_workers=max_workers,
+ timeout=timeout_sec,
+ initializer=initializer,
+ )
+ self.timeout_sec = timeout_sec
+ self.f_build = f_build
+ self.f_export = f_export
+ self._sanity_check()
+
+ def build(self, build_inputs: List[BuilderInput]) -> List[BuilderResult]:
+ results: List[BuilderResult] = []
+ map_result: MapResult
+
+ # Dispatch the build inputs to the worker processes.
+ for map_result in self.pool.map_with_error_catching(
+ lambda x: LocalBuilder._worker_func(*x),
+ [
+ (
+ self.f_build,
+ self.f_export,
+ build_input.mod,
+ build_input.target,
+ )
+ for build_input in build_inputs
+ ],
+ ):
+ if map_result.status == StatusKind.COMPLETE:
+ results.append(BuilderResult(map_result.value, None))
+ elif map_result.status == StatusKind.TIMEOUT:
+ results.append(
+ BuilderResult(
+ None,
+ f"LocalBuilder: Timeout, killed after {self.timeout_sec} seconds",
+ )
+ )
+ elif map_result.status == StatusKind.EXCEPTION:
+ results.append(
+ BuilderResult(
+ None,
+ "LocalBuilder: An exception occurred\n" + str(map_result.value),
+ )
+ )
+ else:
+ raise NotImplementedError(map_result.status)
Review comment:
Then I think we could just raise an exception with a message saying it is unreachable. The purpose is to let developers know we don't need to worry about that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbrookhart commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
mbrookhart commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r712299821
##########
File path: python/tvm/relay/transform/transform.py
##########
@@ -1204,7 +1204,7 @@ def FakeQuantizationToInteger():
x w
| |
dq dq
- \ /
+ \\ /
Review comment:
Nit: Why is your code formatter screwing with my ASCII Art? :grin:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711697976
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
Review comment:
actually we support passing in a function too. will rework the annotation to make it clearer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711699573
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
Review comment:
done in the latest commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbrookhart commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
mbrookhart commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r712512220
##########
File path: python/tvm/relay/transform/transform.py
##########
@@ -1204,7 +1204,7 @@ def FakeQuantizationToInteger():
x w
| |
dq dq
- \ /
+ \\ /
Review comment:
Hmm, I wonder why I didn't see that in CI when I added this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
comaniac commented on pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#issuecomment-923328570
Thanks @junrushao1994 @mbrookhart
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711699657
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
+ initializer: Optional[Callable[[], None]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ max_workers : Optional[int]
+ The maximum number of worker processes to be used.
+ Defaults to number of CPUs.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+ initializer : Optional[Callable[[], None]]
+ The initializer to be used for the worker processes.
+ """
+ super().__init__()
+
+ if max_workers is None:
+ max_workers = cpu_count()
+
+ self.pool = PopenPoolExecutor(
+ max_workers=max_workers,
+ timeout=timeout_sec,
+ initializer=initializer,
+ )
+ self.timeout_sec = timeout_sec
+ self.f_build = f_build
+ self.f_export = f_export
+ self._sanity_check()
+
+ def build(self, build_inputs: List[BuilderInput]) -> List[BuilderResult]:
+ results: List[BuilderResult] = []
+ map_result: MapResult
+
+ # Dispatch the build inputs to the worker processes.
+ for map_result in self.pool.map_with_error_catching(
+ lambda x: LocalBuilder._worker_func(*x),
+ [
+ (
+ self.f_build,
+ self.f_export,
+ build_input.mod,
+ build_input.target,
+ )
+ for build_input in build_inputs
+ ],
+ ):
+ if map_result.status == StatusKind.COMPLETE:
+ results.append(BuilderResult(map_result.value, None))
+ elif map_result.status == StatusKind.TIMEOUT:
+ results.append(
+ BuilderResult(
+ None,
+ f"LocalBuilder: Timeout, killed after {self.timeout_sec} seconds",
+ )
+ )
+ elif map_result.status == StatusKind.EXCEPTION:
+ results.append(
+ BuilderResult(
+ None,
+ "LocalBuilder: An exception occurred\n" + str(map_result.value),
+ )
+ )
+ else:
+ raise NotImplementedError(map_result.status)
+ return results
+
+ def _sanity_check(self) -> None:
+ def _check(
+ f_build: Optional[str],
+ f_export: Optional[str],
+ ) -> None:
+ get_global_func_with_default_on_worker(name=f_build, default=None)
+ get_global_func_with_default_on_worker(name=f_export, default=None)
+
+ value = self.pool.submit(_check, self.f_build, self.f_export)
+ value.result()
+
+ @staticmethod
+ def _worker_func(
+ _f_build: Optional[str],
+ _f_export: Optional[str],
+ mod: IRModule,
+ target: Target,
+ ) -> str:
+ # Step 0. Get the registered functions
+ f_build: Callable[
+ [IRModule, Target],
+ Module,
+ ] = get_global_func_with_default_on_worker(_f_build, default_build)
+ f_export: Callable[
+ [Module],
+ str,
+ ] = get_global_func_with_default_on_worker(_f_export, default_export)
Review comment:
Yeah it is good point too. In the latest commit, I added LocalBuilder.T_BUILD and its corresponding documentation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711697809
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
Review comment:
good point. it should be Optional[str]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711706672
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
+ initializer: Optional[Callable[[], None]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ max_workers : Optional[int]
+ The maximum number of worker processes to be used.
+ Defaults to number of CPUs.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+ initializer : Optional[Callable[[], None]]
+ The initializer to be used for the worker processes.
+ """
+ super().__init__()
+
+ if max_workers is None:
+ max_workers = cpu_count()
Review comment:
My concern is actually the CPU memory consumption. I remember that I was encountering process killed issue due to this back to the time of using AutoTVM. Not sure if this would still be a case tho.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #9044: [Meta Schedule][M3b] Builder
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #9044:
URL: https://github.com/apache/tvm/pull/9044#discussion_r711700131
##########
File path: python/tvm/meta_schedule/builder/local_builder.py
##########
@@ -0,0 +1,224 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Local builder that compile on the local host"""
+import os
+import tempfile
+from typing import Callable, List, Optional
+
+from tvm._ffi import register_func
+from tvm.ir import IRModule
+from tvm.runtime import Module
+from tvm.target import Target
+
+from ...contrib.popen_pool import MapResult, PopenPoolExecutor, StatusKind
+from ..utils import cpu_count, get_global_func_with_default_on_worker
+from .builder import BuilderInput, BuilderResult, PyBuilder
+
+
+class LocalBuilder(PyBuilder):
+ """A builder that builds the given input on local host.
+
+ Parameters
+ ----------
+ pool : PopenPoolExecutor
+ The process pool to run the build.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+
+ Note
+ ----
+ The build function and export function should be registered in the worker process.
+ The worker process is only aware of functions registered in TVM package,
+ if there are extra functions to be registered,
+ please send the registration logic via initializer.
+ """
+
+ pool: PopenPoolExecutor
+ timeout_sec: float
+ f_build: Optional[str]
+ f_export: Optional[str]
+
+ def __init__(
+ self,
+ *,
+ max_workers: Optional[int] = None,
+ timeout_sec: float = 30.0,
+ f_build: str = None,
+ f_export: str = None,
+ initializer: Optional[Callable[[], None]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ max_workers : Optional[int]
+ The maximum number of worker processes to be used.
+ Defaults to number of CPUs.
+ timeout_sec : float
+ The timeout in seconds for the build.
+ f_build : Optional[str]
+ Name of the build function to be used.
+ Defaults to `meta_schedule.builder.default_build`.
+ The signature is Callable[[IRModule, Target], Module].
+ f_export : Optional[str]
+ Name of the export function to be used.
+ Defaults to `meta_schedule.builder.default_export`.
+ The signature is Callable[[Module], str].
+ initializer : Optional[Callable[[], None]]
+ The initializer to be used for the worker processes.
+ """
+ super().__init__()
+
+ if max_workers is None:
+ max_workers = cpu_count()
+
+ self.pool = PopenPoolExecutor(
+ max_workers=max_workers,
+ timeout=timeout_sec,
+ initializer=initializer,
+ )
+ self.timeout_sec = timeout_sec
+ self.f_build = f_build
+ self.f_export = f_export
+ self._sanity_check()
+
+ def build(self, build_inputs: List[BuilderInput]) -> List[BuilderResult]:
+ results: List[BuilderResult] = []
+ map_result: MapResult
+
+ # Dispatch the build inputs to the worker processes.
+ for map_result in self.pool.map_with_error_catching(
+ lambda x: LocalBuilder._worker_func(*x),
+ [
+ (
+ self.f_build,
+ self.f_export,
+ build_input.mod,
+ build_input.target,
+ )
+ for build_input in build_inputs
+ ],
+ ):
+ if map_result.status == StatusKind.COMPLETE:
+ results.append(BuilderResult(map_result.value, None))
+ elif map_result.status == StatusKind.TIMEOUT:
+ results.append(
+ BuilderResult(
+ None,
+ f"LocalBuilder: Timeout, killed after {self.timeout_sec} seconds",
+ )
+ )
+ elif map_result.status == StatusKind.EXCEPTION:
+ results.append(
+ BuilderResult(
+ None,
+ "LocalBuilder: An exception occurred\n" + str(map_result.value),
+ )
+ )
+ else:
+ raise NotImplementedError(map_result.status)
Review comment:
This branch is actually unreachable, given the StatusKind only has 4 possible outcomes:
- RUNNING => it won't appear in our case, because popen pool enforces join semantics
- COMPLETE => handled in the 1st branch
- EXCEPTION => handled in the 3rd branch
- TIMEOUT => handled in the 2nd branch
So actually it doesn't matter whatever we raise, because the branch is unreachable. What do you think is a good default for this case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org