You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by tq...@apache.org on 2024/09/24 03:25:19 UTC
(tvm-site) branch asf-site updated: deploying docs (apache/tvm@dfd9bd581d2d866d552c8e099568c6127aa3f971)
This is an automated email from the ASF dual-hosted git repository.
tqchen pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/tvm-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new d8ed699155 deploying docs (apache/tvm@dfd9bd581d2d866d552c8e099568c6127aa3f971)
d8ed699155 is described below
commit d8ed6991552d207324c77faeeaf234d9ddfffdc2
Author: tvm-bot <95...@users.noreply.github.com>
AuthorDate: Tue Sep 24 03:25:12 2024 +0000
deploying docs (apache/tvm@dfd9bd581d2d866d552c8e099568c6127aa3f971)
---
.../tune_network_arm.py | 1 -
.../tune_network_arm.ipynb | 2 +-
.../tune_network_mali.ipynb | 2 +-
.../tune_network_mali.py | 1 -
.../micro_tvmc.ipynb | 2 +-
.../tune_network_x86.ipynb | 2 +-
.../tune_network_cuda.ipynb | 2 +-
.../tune_network_x86.py | 1 -
.../tune_network_cuda.py | 1 -
.../eb483c672b88006c331115968e0ffd9b/micro_tvmc.py | 2 +-
docs/_sources/arch/benchmark.rst.txt | 137 ---
docs/_sources/arch/convert_layout.rst.txt | 269 -----
docs/_sources/arch/frontend/tensorflow.rst.txt | 254 -----
docs/_sources/arch/hybrid_script.rst.txt | 100 --
docs/_sources/arch/index.rst.txt | 218 ++--
docs/_sources/arch/inferbound.rst.txt | 763 --------------
docs/_sources/arch/microtvm_design.rst.txt | 357 -------
docs/_sources/arch/microtvm_project_api.rst.txt | 150 ---
docs/_sources/arch/model_library_format.rst.txt | 171 ----
docs/_sources/arch/relay_intro.rst.txt | 206 ----
docs/_sources/arch/relay_op_strategy.rst.txt | 282 -----
docs/_sources/arch/virtual_machine.rst.txt | 410 --------
docs/_sources/deep_dive/relax/index.rst.txt | 2 +-
.../relax/tutorials/sg_execution_times.rst.txt | 6 +-
docs/_sources/deep_dive/tensor_ir/index.rst.txt | 2 +-
.../tensor_ir/tutorials/sg_execution_times.rst.txt | 4 +-
.../tensor_ir/tutorials/tir_transformation.rst.txt | 6 +-
.../dev/tutorial/codebase_walkthrough.rst.txt | 2 +-
.../tutorials/sg_execution_times.rst.txt | 6 +-
.../how_to/compile_models/from_darknet.rst.txt | 2 +-
.../how_to/compile_models/from_keras.rst.txt | 2 +-
.../how_to/compile_models/from_oneflow.rst.txt | 2 +-
.../how_to/compile_models/from_paddle.rst.txt | 2 +-
.../how_to/compile_models/from_pytorch.rst.txt | 2 +-
.../how_to/compile_models/from_tensorflow.rst.txt | 2 +-
.../compile_models/sg_execution_times.rst.txt | 20 +-
.../deploy_models/deploy_model_on_adreno.rst.txt | 4 +-
.../deploy_model_on_adreno_tvmc.rst.txt | 2 +-
.../deploy_models/deploy_model_on_android.rst.txt | 2 +-
.../deploy_object_detection_pytorch.rst.txt | 4 +-
.../deploy_models/deploy_prequantized.rst.txt | 6 +-
.../deploy_prequantized_tflite.rst.txt | 2 +-
.../deploy_models/sg_execution_times.rst.txt | 18 +-
.../how_to/extend_tvm/sg_execution_times.rst.txt | 8 +-
.../how_to/extend_tvm/use_pass_instrument.rst.txt | 16 +-
.../optimize_operators/opt_conv_cuda.rst.txt | 2 +-
.../optimize_operators/opt_conv_tensorcore.rst.txt | 2 +-
.../how_to/optimize_operators/opt_gemm.rst.txt | 16 +-
.../optimize_operators/sg_execution_times.rst.txt | 8 +-
.../sg_execution_times.rst.txt | 14 +-
.../tune_network_arm.rst.txt | 27 +-
.../tune_network_cuda.rst.txt | 27 +-
.../tune_network_mali.rst.txt | 25 +-
.../tune_network_x86.rst.txt | 27 +-
.../tune_with_autotvm/sg_execution_times.rst.txt | 6 +-
.../tune_with_autotvm/tune_conv2d_cuda.rst.txt | 2 +-
.../tutorials/cross_compilation_and_rpc.rst.txt | 2 +-
.../_sources/how_to/tutorials/optimize_llm.rst.txt | 2 +-
.../how_to/tutorials/sg_execution_times.rst.txt | 10 +-
.../work_with_microtvm/micro_autotune.rst.txt | 18 +-
.../work_with_microtvm/micro_pytorch.rst.txt | 4 +-
.../how_to/work_with_microtvm/micro_train.rst.txt | 16 +-
.../how_to/work_with_microtvm/micro_tvmc.rst.txt | 2 +-
.../work_with_microtvm/sg_execution_times.rst.txt | 14 +-
.../work_with_relay/sg_execution_times.rst.txt | 8 +-
.../work_with_schedules/sg_execution_times.rst.txt | 10 +-
docs/_sources/index.rst.txt | 2 +-
docs/_sources/reference/langref/relay_expr.rst.txt | 4 +-
docs/_sources/topic/microtvm/index.rst.txt | 7 -
.../tutorials/frontend/deploy_detection.rst.txt | 4 +-
.../tutorials/frontend/sg_execution_times.rst.txt | 4 +-
.../tutorials/optimize/sg_execution_times.rst.txt | 6 +-
.../topic/vta/tutorials/sg_execution_times.rst.txt | 6 +-
.../tutorial/auto_scheduler_matmul_x86.rst.txt | 11 +-
docs/_sources/tutorial/autotvm_matmul_x86.rst.txt | 20 +-
docs/_sources/tutorial/autotvm_relay_x86.rst.txt | 57 +-
docs/_sources/tutorial/intro_topi.rst.txt | 2 +-
docs/_sources/tutorial/sg_execution_times.rst.txt | 20 +-
.../tutorial/tensor_expr_get_started.rst.txt | 44 +-
docs/arch/benchmark.html | 666 ------------
docs/arch/convert_layout.html | 706 -------------
docs/arch/debugger.html | 48 +-
docs/arch/device_target_interactions.html | 44 +-
docs/arch/frontend/tensorflow.html | 709 -------------
docs/arch/hybrid_script.html | 551 ----------
docs/arch/index.html | 274 ++---
docs/arch/inferbound.html | 1080 --------------------
.../arch/introduction_to_module_serialization.html | 48 +-
docs/arch/microtvm_design.html | 830 ---------------
docs/arch/microtvm_project_api.html | 610 -----------
docs/arch/model_library_format.html | 652 ------------
docs/arch/pass_infra.html | 47 +-
docs/arch/relay_intro.html | 645 ------------
docs/arch/relay_op_strategy.html | 708 -------------
docs/arch/runtime.html | 44 +-
docs/arch/runtimes/vulkan.html | 44 +-
docs/arch/virtual_machine.html | 818 ---------------
docs/commit_hash | 2 +-
docs/contribute/ci.html | 2 +-
docs/contribute/code_guide.html | 2 +-
docs/contribute/code_review.html | 2 +-
docs/contribute/committer_guide.html | 2 +-
docs/contribute/community.html | 2 +-
docs/contribute/document.html | 2 +-
docs/contribute/error_handling.html | 2 +-
docs/contribute/git_howto.html | 2 +-
docs/contribute/index.html | 2 +-
docs/contribute/pull_request.html | 2 +-
docs/contribute/release_process.html | 2 +-
docs/deep_dive/relax/abstraction.html | 2 +-
docs/deep_dive/relax/index.html | 4 +-
docs/deep_dive/relax/learning.html | 2 +-
docs/deep_dive/relax/tutorials/index.html | 2 +-
docs/deep_dive/relax/tutorials/relax_creation.html | 2 +-
.../relax/tutorials/relax_transformation.html | 2 +-
.../relax/tutorials/sg_execution_times.html | 8 +-
docs/deep_dive/tensor_ir/abstraction.html | 2 +-
docs/deep_dive/tensor_ir/index.html | 8 +-
docs/deep_dive/tensor_ir/learning.html | 2 +-
docs/deep_dive/tensor_ir/tutorials/index.html | 2 +-
.../tensor_ir/tutorials/sg_execution_times.html | 6 +-
.../tensor_ir/tutorials/tir_creation.html | 2 +-
.../tensor_ir/tutorials/tir_transformation.html | 8 +-
docs/dev/how_to/debugging_tvm.html | 2 +-
docs/dev/how_to/how_to.html | 2 +-
docs/dev/how_to/relay_add_op.html | 2 +-
docs/dev/how_to/relay_add_pass.html | 2 +-
docs/dev/how_to/relay_bring_your_own_codegen.html | 2 +-
docs/dev/tutorial/codebase_walkthrough.html | 4 +-
docs/dev/tutorial/index.html | 2 +-
docs/errors.html | 6 +-
docs/faq.html | 2 +-
docs/genindex.html | 2 +-
docs/get_started/overview.html | 2 +-
docs/get_started/tutorials/index.html | 2 +-
docs/get_started/tutorials/ir_module.html | 2 +-
docs/get_started/tutorials/quick_start.html | 2 +-
docs/get_started/tutorials/sg_execution_times.html | 8 +-
docs/how_to/compile_models/from_coreml.html | 2 +-
docs/how_to/compile_models/from_darknet.html | 4 +-
docs/how_to/compile_models/from_keras.html | 4 +-
docs/how_to/compile_models/from_oneflow.html | 20 +-
docs/how_to/compile_models/from_onnx.html | 2 +-
docs/how_to/compile_models/from_paddle.html | 4 +-
docs/how_to/compile_models/from_pytorch.html | 14 +-
docs/how_to/compile_models/from_tensorflow.html | 4 +-
docs/how_to/compile_models/from_tflite.html | 2 +-
docs/how_to/compile_models/index.html | 2 +-
docs/how_to/compile_models/sg_execution_times.html | 22 +-
docs/how_to/deploy/adreno.html | 2 +-
docs/how_to/deploy/android.html | 2 +-
docs/how_to/deploy/arm_compute_lib.html | 2 +-
docs/how_to/deploy/bnns.html | 2 +-
docs/how_to/deploy/cpp_deploy.html | 2 +-
docs/how_to/deploy/hls.html | 2 +-
docs/how_to/deploy/index.html | 2 +-
docs/how_to/deploy/integrate.html | 2 +-
docs/how_to/deploy/mrvl.html | 2 +-
docs/how_to/deploy/tensorrt.html | 2 +-
docs/how_to/deploy/vitis_ai.html | 2 +-
.../deploy_models/deploy_model_on_adreno.html | 6 +-
.../deploy_models/deploy_model_on_adreno_tvmc.html | 28 +-
.../deploy_models/deploy_model_on_android.html | 4 +-
.../how_to/deploy_models/deploy_model_on_nano.html | 2 +-
.../how_to/deploy_models/deploy_model_on_rasp.html | 2 +-
.../deploy_object_detection_pytorch.html | 25 +-
docs/how_to/deploy_models/deploy_prequantized.html | 10 +-
.../deploy_models/deploy_prequantized_tflite.html | 4 +-
docs/how_to/deploy_models/deploy_sparse.html | 2 +-
docs/how_to/deploy_models/index.html | 2 +-
docs/how_to/deploy_models/sg_execution_times.html | 24 +-
docs/how_to/dev/index.html | 2 +-
docs/how_to/dev/pytest_target_parametrization.html | 2 +-
docs/how_to/dev/setup_rpc_system.html | 2 +-
docs/how_to/extend_tvm/index.html | 2 +-
docs/how_to/extend_tvm/low_level_custom_pass.html | 2 +-
docs/how_to/extend_tvm/sg_execution_times.html | 10 +-
docs/how_to/extend_tvm/use_pass_infra.html | 2 +-
docs/how_to/extend_tvm/use_pass_instrument.html | 18 +-
docs/how_to/legacy_index.html | 2 +-
docs/how_to/optimize_operators/index.html | 2 +-
docs/how_to/optimize_operators/opt_conv_cuda.html | 4 +-
.../optimize_operators/opt_conv_tensorcore.html | 4 +-
docs/how_to/optimize_operators/opt_gemm.html | 18 +-
.../optimize_operators/sg_execution_times.html | 10 +-
docs/how_to/profile/index.html | 2 +-
docs/how_to/profile/papi.html | 2 +-
docs/how_to/tune_with_autoscheduler/index.html | 2 +-
.../sg_execution_times.html | 16 +-
.../tune_conv2d_layer_cuda.html | 2 +-
.../tune_with_autoscheduler/tune_network_arm.html | 5 +-
.../tune_with_autoscheduler/tune_network_cuda.html | 9 +-
.../tune_with_autoscheduler/tune_network_mali.html | 5 +-
.../tune_with_autoscheduler/tune_network_x86.html | 9 +-
.../tune_with_autoscheduler/tune_sparse_x86.html | 2 +-
docs/how_to/tune_with_autotvm/index.html | 2 +-
.../tune_with_autotvm/sg_execution_times.html | 8 +-
.../how_to/tune_with_autotvm/tune_conv2d_cuda.html | 4 +-
docs/how_to/tune_with_autotvm/tune_relay_arm.html | 2 +-
docs/how_to/tune_with_autotvm/tune_relay_cuda.html | 2 +-
.../tune_with_autotvm/tune_relay_mobile_gpu.html | 2 +-
docs/how_to/tune_with_autotvm/tune_relay_x86.html | 2 +-
.../tutorials/cross_compilation_and_rpc.html | 4 +-
docs/how_to/tutorials/customize_opt.html | 2 +-
docs/how_to/tutorials/e2e_opt_model.html | 2 +-
docs/how_to/tutorials/index.html | 2 +-
docs/how_to/tutorials/optimize_llm.html | 4 +-
docs/how_to/tutorials/sg_execution_times.html | 16 +-
docs/how_to/work_with_microtvm/index.html | 2 +-
docs/how_to/work_with_microtvm/micro_aot.html | 2 +-
docs/how_to/work_with_microtvm/micro_autotune.html | 20 +-
.../work_with_microtvm/micro_custom_ide.html | 2 +-
docs/how_to/work_with_microtvm/micro_ethosu.html | 2 +-
.../work_with_microtvm/micro_mlperftiny.html | 2 +-
docs/how_to/work_with_microtvm/micro_pytorch.html | 6 +-
docs/how_to/work_with_microtvm/micro_tflite.html | 2 +-
docs/how_to/work_with_microtvm/micro_train.html | 18 +-
docs/how_to/work_with_microtvm/micro_tvmc.html | 4 +-
.../work_with_microtvm/sg_execution_times.html | 16 +-
docs/how_to/work_with_relay/build_gcn.html | 2 +-
docs/how_to/work_with_relay/index.html | 2 +-
.../how_to/work_with_relay/sg_execution_times.html | 10 +-
.../how_to/work_with_relay/using_external_lib.html | 2 +-
.../work_with_relay/using_pipeline_executor.html | 2 +-
docs/how_to/work_with_relay/using_relay_viz.html | 2 +-
docs/how_to/work_with_schedules/extern_op.html | 2 +-
docs/how_to/work_with_schedules/index.html | 2 +-
docs/how_to/work_with_schedules/reduction.html | 2 +-
docs/how_to/work_with_schedules/scan.html | 2 +-
.../work_with_schedules/schedule_primitives.html | 2 +-
.../work_with_schedules/sg_execution_times.html | 12 +-
docs/how_to/work_with_schedules/tedd.html | 2 +-
docs/how_to/work_with_schedules/tensorize.html | 2 +-
docs/how_to/work_with_schedules/tuple_inputs.html | 2 +-
docs/index.html | 18 +-
docs/install/docker.html | 2 +-
docs/install/from_source.html | 2 +-
docs/install/index.html | 2 +-
docs/objects.inv | Bin 31716 -> 31158 bytes
docs/py-modindex.html | 2 +-
docs/reference/api/links.html | 2 +-
docs/reference/api/python/auto_scheduler.html | 6 +-
docs/reference/api/python/autotvm.html | 2 +-
docs/reference/api/python/contrib.html | 2 +-
docs/reference/api/python/dlight.html | 2 +-
docs/reference/api/python/driver.html | 2 +-
docs/reference/api/python/error.html | 2 +-
docs/reference/api/python/graph_executor.html | 2 +-
docs/reference/api/python/index.html | 2 +-
docs/reference/api/python/instrument.html | 2 +-
docs/reference/api/python/ir.html | 2 +-
docs/reference/api/python/meta_schedule.html | 2 +-
docs/reference/api/python/micro.html | 2 +-
docs/reference/api/python/relax/analysis.html | 2 +-
docs/reference/api/python/relax/block_builder.html | 2 +-
docs/reference/api/python/relax/frontend.html | 2 +-
docs/reference/api/python/relax/op.html | 2 +-
docs/reference/api/python/relax/relax.html | 4 +-
docs/reference/api/python/relax/transform.html | 2 +-
docs/reference/api/python/relay/analysis.html | 2 +-
docs/reference/api/python/relay/backend.html | 2 +-
.../api/python/relay/dataflow_pattern.html | 2 +-
docs/reference/api/python/relay/frontend.html | 2 +-
docs/reference/api/python/relay/image.html | 2 +-
docs/reference/api/python/relay/index.html | 2 +-
docs/reference/api/python/relay/nn.html | 2 +-
docs/reference/api/python/relay/testing.html | 2 +-
docs/reference/api/python/relay/transform.html | 2 +-
docs/reference/api/python/relay/vision.html | 2 +-
docs/reference/api/python/rpc.html | 2 +-
docs/reference/api/python/runtime/disco.html | 2 +-
docs/reference/api/python/runtime/ndarray.html | 2 +-
docs/reference/api/python/runtime/profiling.html | 2 +-
docs/reference/api/python/runtime/relax_vm.html | 2 +-
docs/reference/api/python/runtime/runtime.html | 2 +-
docs/reference/api/python/target.html | 2 +-
docs/reference/api/python/te.html | 2 +-
docs/reference/api/python/tir/analysis.html | 2 +-
docs/reference/api/python/tir/schedule.html | 2 +-
docs/reference/api/python/tir/stmt_functor.html | 2 +-
docs/reference/api/python/tir/tir.html | 2 +-
docs/reference/api/python/tir/transform.html | 2 +-
docs/reference/api/python/topi.html | 2 +-
docs/reference/api/python/transform.html | 2 +-
.../api/typedoc/classes/ArtifactCache.html | 14 +-
.../typedoc/classes/ArtifactIndexedDBCache.html | 16 +-
docs/reference/api/typedoc/classes/DLDataType.html | 14 +-
docs/reference/api/typedoc/classes/DLDevice.html | 12 +-
docs/reference/api/typedoc/classes/Instance.html | 116 +--
.../classes/LinearCongruentialGenerator.html | 16 +-
docs/reference/api/typedoc/classes/Module.html | 12 +-
docs/reference/api/typedoc/classes/NDArray.html | 30 +-
docs/reference/api/typedoc/classes/RPCServer.html | 16 +-
docs/reference/api/typedoc/classes/Scalar.html | 8 +-
docs/reference/api/typedoc/classes/TVMArray.html | 16 +-
docs/reference/api/typedoc/classes/TVMObject.html | 12 +-
.../api/typedoc/classes/VirtualMachine.html | 10 +-
.../classes/_internal_.AsyncifyHandler.html | 12 +-
.../classes/_internal_.CachedCallStack.html | 36 +-
.../classes/_internal_.CanvasRenderManager.html | 10 +-
.../typedoc/classes/_internal_.Environment.html | 14 +-
.../api/typedoc/classes/_internal_.FFILibrary.html | 22 +-
.../api/typedoc/classes/_internal_.Memory.html | 36 +-
.../typedoc/classes/_internal_.PackedFuncCell.html | 8 +-
.../typedoc/classes/_internal_.RuntimeContext.html | 60 +-
.../api/typedoc/classes/_internal_.TVMString.html | 14 +-
.../typedoc/classes/_internal_.WebGPUContext.html | 28 +-
.../enums/_internal_.AsyncifyStateKind.html | 8 +-
.../typedoc/enums/_internal_.RPCServerState.html | 14 +-
docs/reference/api/typedoc/functions/assert.html | 2 +-
.../api/typedoc/functions/createPolyfillWASI.html | 2 +-
.../api/typedoc/functions/deleteNDArrayCache.html | 2 +-
.../api/typedoc/functions/detectGPUDevice.html | 2 +-
.../api/typedoc/functions/hasNDArrayInCache.html | 2 +-
.../api/typedoc/functions/instantiate.html | 2 +-
docs/reference/api/typedoc/functions/wasmPath.html | 2 +-
.../typedoc/interfaces/ArtifactCacheTemplate.html | 10 +-
.../api/typedoc/interfaces/Disposable.html | 4 +-
.../typedoc/interfaces/GPUDeviceDetectOutput.html | 8 +-
.../api/typedoc/interfaces/InitProgressReport.html | 8 +-
.../api/typedoc/interfaces/LibraryProvider.html | 6 +-
.../interfaces/_internal_.FunctionInfo.html | 8 +-
.../interfaces/_internal_.NDArrayCacheEntry.html | 14 +-
.../interfaces/_internal_.NDArrayShardEntry.html | 10 +-
.../api/typedoc/types/InitProgressCallback.html | 2 +-
docs/reference/api/typedoc/types/PackedFunc.html | 2 +-
.../typedoc/types/_internal_.AsyncPackedFunc.html | 2 +-
.../types/_internal_.FObjectConstructor.html | 2 +-
.../types/_internal_.FTVMWasmAllocSpace.html | 2 +-
.../types/_internal_.FTVMWasmFreeSpace.html | 2 +-
.../types/_internal_.FTVMWasmPackedCFunc.html | 2 +-
.../api/typedoc/types/_internal_.Pointer.html | 2 +-
.../typedoc/types/_internal_.TVMObjectBase.html | 2 +-
docs/reference/langref/hybrid_script.html | 6 +-
docs/reference/langref/index.html | 2 +-
docs/reference/langref/relay_adt.html | 2 +-
docs/reference/langref/relay_expr.html | 6 +-
docs/reference/langref/relay_op.html | 2 +-
docs/reference/langref/relay_pattern.html | 2 +-
docs/reference/langref/relay_type.html | 2 +-
docs/reference/publications.html | 2 +-
docs/reference/security.html | 2 +-
docs/search.html | 2 +-
docs/searchindex.js | 2 +-
docs/topic/microtvm/index.html | 11 +-
docs/topic/vta/dev/config.html | 2 +-
docs/topic/vta/dev/hardware.html | 2 +-
docs/topic/vta/dev/index.html | 2 +-
docs/topic/vta/index.html | 2 +-
docs/topic/vta/install.html | 2 +-
.../vta/tutorials/frontend/deploy_detection.html | 6 +-
docs/topic/vta/tutorials/frontend/index.html | 2 +-
.../vta/tutorials/frontend/sg_execution_times.html | 6 +-
docs/topic/vta/tutorials/index.html | 2 +-
docs/topic/vta/tutorials/matrix_multiply.html | 2 +-
.../vta/tutorials/optimize/convolution_opt.html | 2 +-
docs/topic/vta/tutorials/optimize/index.html | 2 +-
.../tutorials/optimize/matrix_multiply_opt.html | 2 +-
.../vta/tutorials/optimize/sg_execution_times.html | 8 +-
docs/topic/vta/tutorials/sg_execution_times.html | 8 +-
docs/topic/vta/tutorials/vta_get_started.html | 2 +-
docs/tutorial/auto_scheduler_matmul_x86.html | 9 +-
docs/tutorial/autotvm_matmul_x86.html | 22 +-
docs/tutorial/autotvm_relay_x86.html | 267 +++--
docs/tutorial/index.html | 2 +-
docs/tutorial/intro_topi.html | 4 +-
docs/tutorial/introduction.html | 2 +-
docs/tutorial/relay_quick_start.html | 2 +-
docs/tutorial/sg_execution_times.html | 22 +-
docs/tutorial/tensor_expr_get_started.html | 46 +-
docs/tutorial/tensor_ir_blitz_course.html | 2 +-
docs/tutorial/tvmc_command_line_driver.html | 2 +-
docs/tutorial/tvmc_python.html | 2 +-
docs/tutorial/uma.html | 2 +-
374 files changed, 1517 insertions(+), 12781 deletions(-)
diff --git a/docs/_downloads/17b139d609f9480c7eeda2da1f90f28c/tune_network_arm.py b/docs/_downloads/17b139d609f9480c7eeda2da1f90f28c/tune_network_arm.py
index d795c3aba2..e4edf03335 100644
--- a/docs/_downloads/17b139d609f9480c7eeda2da1f90f28c/tune_network_arm.py
+++ b/docs/_downloads/17b139d609f9480c7eeda2da1f90f28c/tune_network_arm.py
@@ -70,7 +70,6 @@ from tvm.contrib.utils import tempdir
# with any layout, we found the best performance is typically achieved with NHWC layout.
# We also implemented more optimizations for NHWC layout with the auto-scheduler.
# So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
-# You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
def get_network(name, batch_size, layout="NHWC", dtype="float32", use_sparse=False):
diff --git a/docs/_downloads/4dc30a43f3a6aa3ed4bc3077ad35ff70/tune_network_arm.ipynb b/docs/_downloads/4dc30a43f3a6aa3ed4bc3077ad35ff70/tune_network_arm.ipynb
index 7e2a9de703..2e7285cad6 100644
--- a/docs/_downloads/4dc30a43f3a6aa3ed4bc3077ad35ff70/tune_network_arm.ipynb
+++ b/docs/_downloads/4dc30a43f3a6aa3ed4bc3077ad35ff70/tune_network_arm.ipynb
@@ -33,7 +33,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Define a Network\nFirst, we need to define the network with relay frontend API.\nWe can load some pre-defined network from :code:`tvm.relay.testing`.\nWe can also load models from MXNet, ONNX, PyTorch, and TensorFlow\n(see `front end tutorials<tutorial-frontend>`).\n\nFor convolutional neural networks, although auto-scheduler can work correctly\nwith any layout, we found the best performance is typically achieved with NHWC layout.\nWe also implemented more optimizations for N [...]
+ "## Define a Network\nFirst, we need to define the network with relay frontend API.\nWe can load some pre-defined network from :code:`tvm.relay.testing`.\nWe can also load models from MXNet, ONNX, PyTorch, and TensorFlow\n(see `front end tutorials<tutorial-frontend>`).\n\nFor convolutional neural networks, although auto-scheduler can work correctly\nwith any layout, we found the best performance is typically achieved with NHWC layout.\nWe also implemented more optimizations for N [...]
]
},
{
diff --git a/docs/_downloads/5e4e499c097b16a90c517e630502253a/tune_network_mali.ipynb b/docs/_downloads/5e4e499c097b16a90c517e630502253a/tune_network_mali.ipynb
index a10a56fe85..f4e1d9370c 100644
--- a/docs/_downloads/5e4e499c097b16a90c517e630502253a/tune_network_mali.ipynb
+++ b/docs/_downloads/5e4e499c097b16a90c517e630502253a/tune_network_mali.ipynb
@@ -33,7 +33,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Define a Network\nFirst, we need to define the network with relay frontend API.\nWe can load some pre-defined network from :code:`tvm.relay.testing`.\nWe can also load models from MXNet, ONNX, PyTorch, and TensorFlow\n(see `front end tutorials<tutorial-frontend>`).\n\nFor convolutional neural networks, although auto-scheduler can work correctly\nwith any layout, we found the best performance is typically achieved with NHWC layout.\nWe also implemented more optimizations for N [...]
+ "## Define a Network\nFirst, we need to define the network with relay frontend API.\nWe can load some pre-defined network from :code:`tvm.relay.testing`.\nWe can also load models from MXNet, ONNX, PyTorch, and TensorFlow\n(see `front end tutorials<tutorial-frontend>`).\n\nFor convolutional neural networks, although auto-scheduler can work correctly\nwith any layout, we found the best performance is typically achieved with NHWC layout.\nWe also implemented more optimizations for N [...]
]
},
{
diff --git a/docs/_downloads/67bf7dd99bcfb837cf3e8b461a5eeb48/tune_network_mali.py b/docs/_downloads/67bf7dd99bcfb837cf3e8b461a5eeb48/tune_network_mali.py
index 15f3379013..3120c30cef 100644
--- a/docs/_downloads/67bf7dd99bcfb837cf3e8b461a5eeb48/tune_network_mali.py
+++ b/docs/_downloads/67bf7dd99bcfb837cf3e8b461a5eeb48/tune_network_mali.py
@@ -67,7 +67,6 @@ from tvm.contrib import graph_executor
# with any layout, we found the best performance is typically achieved with NHWC layout.
# We also implemented more optimizations for NHWC layout with the auto-scheduler.
# So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
-# You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
def get_network(name, batch_size, layout="NHWC", dtype="float32"):
diff --git a/docs/_downloads/6e511f5a8ddbf12f2fca2dfadc0cc4a9/micro_tvmc.ipynb b/docs/_downloads/6e511f5a8ddbf12f2fca2dfadc0cc4a9/micro_tvmc.ipynb
index 523046a867..a5a1094259 100644
--- a/docs/_downloads/6e511f5a8ddbf12f2fca2dfadc0cc4a9/micro_tvmc.ipynb
+++ b/docs/_downloads/6e511f5a8ddbf12f2fca2dfadc0cc4a9/micro_tvmc.ipynb
@@ -72,7 +72,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Compiling a TFLite model to a Model Library Format\n\n Model Library Format (MLF) is an output format that TVM provides for micro targets. MLF is a tarball\n containing a file for each piece of the TVM compiler output which can be used on micro targets outside\n TVM environment. Read more about `Model Library Format <model_library_format>`.\n\n Here, we generate a MLF file for ``qemu_x86`` Zephyr board. You can chooses `aot` or `graph` executor type\n to run this tutorial, h [...]
+ "### Compiling a TFLite model to a Model Library Format\n\n Model Library Format (MLF) is an output format that TVM provides for micro targets. MLF is a tarball\n containing a file for each piece of the TVM compiler output which can be used on micro targets outside\n TVM environment.\n\n Here, we generate a MLF file for ``qemu_x86`` Zephyr board. You can chooses `aot` or `graph` executor type\n to run this tutorial, however, we recommend to use `aot` for microTVM targets since `a [...]
]
},
{
diff --git a/docs/_downloads/ad2a7f55d615d188ad664d56696815a6/tune_network_x86.ipynb b/docs/_downloads/ad2a7f55d615d188ad664d56696815a6/tune_network_x86.ipynb
index d8522c5d39..13de697244 100644
--- a/docs/_downloads/ad2a7f55d615d188ad664d56696815a6/tune_network_x86.ipynb
+++ b/docs/_downloads/ad2a7f55d615d188ad664d56696815a6/tune_network_x86.ipynb
@@ -33,7 +33,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Define a Network\nFirst, we need to define the network with relay frontend API.\nWe can load some pre-defined network from :code:`tvm.relay.testing`.\nWe can also load models from MXNet, ONNX, PyTorch, and TensorFlow\n(see `front end tutorials<tutorial-frontend>`).\n\nFor convolutional neural networks, although auto-scheduler can work correctly\nwith any layout, we found the best performance is typically achieved with NHWC layout.\nWe also implemented more optimizations for N [...]
+ "## Define a Network\nFirst, we need to define the network with relay frontend API.\nWe can load some pre-defined network from :code:`tvm.relay.testing`.\nWe can also load models from MXNet, ONNX, PyTorch, and TensorFlow\n(see `front end tutorials<tutorial-frontend>`).\n\nFor convolutional neural networks, although auto-scheduler can work correctly\nwith any layout, we found the best performance is typically achieved with NHWC layout.\nWe also implemented more optimizations for N [...]
]
},
{
diff --git a/docs/_downloads/af264436d049e3cd84803b67b6620b63/tune_network_cuda.ipynb b/docs/_downloads/af264436d049e3cd84803b67b6620b63/tune_network_cuda.ipynb
index 29906beb74..4acb5f5918 100644
--- a/docs/_downloads/af264436d049e3cd84803b67b6620b63/tune_network_cuda.ipynb
+++ b/docs/_downloads/af264436d049e3cd84803b67b6620b63/tune_network_cuda.ipynb
@@ -33,7 +33,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Define a Network\nFirst, we need to define the network with relay frontend API.\nWe can load some pre-defined network from :code:`tvm.relay.testing`.\nWe can also load models from MXNet, ONNX, PyTorch, and TensorFlow\n(see `front end tutorials<tutorial-frontend>`).\n\nFor convolutional neural networks, although auto-scheduler can work correctly\nwith any layout, we found the best performance is typically achieved with NHWC layout.\nWe also implemented more optimizations for N [...]
+ "## Define a Network\nFirst, we need to define the network with relay frontend API.\nWe can load some pre-defined network from :code:`tvm.relay.testing`.\nWe can also load models from MXNet, ONNX, PyTorch, and TensorFlow\n(see `front end tutorials<tutorial-frontend>`).\n\nFor convolutional neural networks, although auto-scheduler can work correctly\nwith any layout, we found the best performance is typically achieved with NHWC layout.\nWe also implemented more optimizations for N [...]
]
},
{
diff --git a/docs/_downloads/e416b94ca1090b0897c0f6e0df95b911/tune_network_x86.py b/docs/_downloads/e416b94ca1090b0897c0f6e0df95b911/tune_network_x86.py
index 169567122f..43314a4b0a 100644
--- a/docs/_downloads/e416b94ca1090b0897c0f6e0df95b911/tune_network_x86.py
+++ b/docs/_downloads/e416b94ca1090b0897c0f6e0df95b911/tune_network_x86.py
@@ -67,7 +67,6 @@ from tvm.contrib import graph_executor
# with any layout, we found the best performance is typically achieved with NHWC layout.
# We also implemented more optimizations for NHWC layout with the auto-scheduler.
# So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
-# You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
def get_network(name, batch_size, layout="NHWC", dtype="float32", use_sparse=False):
diff --git a/docs/_downloads/eafe360d52540634c9eea0fa89e804bd/tune_network_cuda.py b/docs/_downloads/eafe360d52540634c9eea0fa89e804bd/tune_network_cuda.py
index 1f8c0cc13a..f11aef253f 100644
--- a/docs/_downloads/eafe360d52540634c9eea0fa89e804bd/tune_network_cuda.py
+++ b/docs/_downloads/eafe360d52540634c9eea0fa89e804bd/tune_network_cuda.py
@@ -64,7 +64,6 @@ from tvm.contrib import graph_executor
# with any layout, we found the best performance is typically achieved with NHWC layout.
# We also implemented more optimizations for NHWC layout with the auto-scheduler.
# So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
-# You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
def get_network(name, batch_size, layout="NHWC", dtype="float32"):
diff --git a/docs/_downloads/eb483c672b88006c331115968e0ffd9b/micro_tvmc.py b/docs/_downloads/eb483c672b88006c331115968e0ffd9b/micro_tvmc.py
index b0eeebdbf0..7f429a08ee 100644
--- a/docs/_downloads/eb483c672b88006c331115968e0ffd9b/micro_tvmc.py
+++ b/docs/_downloads/eb483c672b88006c331115968e0ffd9b/micro_tvmc.py
@@ -90,7 +90,7 @@ You need to install python and Zephyr dependencies before processing with this t
#
# Model Library Format (MLF) is an output format that TVM provides for micro targets. MLF is a tarball
# containing a file for each piece of the TVM compiler output which can be used on micro targets outside
-# TVM environment. Read more about :ref:`Model Library Format <model_library_format>`.
+# TVM environment.
#
# Here, we generate a MLF file for ``qemu_x86`` Zephyr board. You can chooses `aot` or `graph` executor type
# to run this tutorial, however, we recommend to use `aot` for microTVM targets since `aot` uses ahead of time
diff --git a/docs/_sources/arch/benchmark.rst.txt b/docs/_sources/arch/benchmark.rst.txt
deleted file mode 100644
index 8217a4feb7..0000000000
--- a/docs/_sources/arch/benchmark.rst.txt
+++ /dev/null
@@ -1,137 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-********************************
-Benchmark Performance Log Format
-********************************
-This page details schema v0.1 for a unified benchmark log format. This schema will allow easier cross-references with other frameworks/runs, experiment reproduction, data for nightly perf regression, and the separation of logging/visualization efforts.
-
-Log Format Overview
-~~~~~~~~~~~~~~~~~~~
-
-For simplicity, we suggest prioritizing the fields `workload`, `engine`, `hardware` `runtime_ms_mean`, and `runtime_ms_std`. For finer-grained logging, one may additionally propagate the `*_config` fields.
-
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| header | examples | category | notes/justification |
-+=======================+==============================================================================================================================================================================+==============+==============================================================================+
-| workload | resnet-18 | workload | name of workload |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| engine | "tvm" / "onnxruntime" | compiler | |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| hardware | "gcp-c2-standard-16" | hardware | descriptor of target hardware environment |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| runtime_ms_mean | 12.452 | statistics | |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| runtime_ms_std | 5.3 | statistics | |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| timestamp | 1572282699.6 | metadata | indicates when this record is logged |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| schema\_version | "0.1" | metadata | ensure reproducibility as we iterate on this schema |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| metadata | { "docker\_tag":"gcr.io/.../0a680", ... } | metadata | ``docker_tag`` is optional |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| workload\_args | {“input\_name”: "Input3", “input\_shape”: [list\_of\_shape], “data\_layout”: NHCW} | workload | |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| workload\_metadata | {"class": "vision","doc\_url": "``https://github.com/.../README.md``", "opset": 7,"type": "body\_analysis","url": "``https://onnxzoo...ferplus.tar.gz``", "md5": "07fc7..."} | workload | source of workload |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| engine\_version | "1.0.5" | compiler | use semvar format |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| engine\_config | {“llvm”: “llvm-8”, “nvcc”: 10.1, "accelerator": "MLAS", "relay_opt_level": 3, "tvm_target":"llvm -mcpu=cascadelake"} | compiler | fields are optionally specified |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| compilation\_config | {"opt_level": 3, "layer_schedules":[]/ <SHA_to_schedules>} | compiler | fields are optionally specified |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| software\_config | {"os": "ubuntu:18.04","pip": { "docker": "4.1.0", "gitpython": "3.0.4", "numpy": "1.17.4", "onnx": "1.6.0"}, “cudnn”: “cudnn-8”, "cuda_driver”: “480.10.1”} | backend | env dependency list |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| runtime\_config | {"num_cpu_threads": 3} | backend | info on non-hardware, non-software metadata |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| hardware\_config | {"cpu_count": 16, "cloud_machine_type":"c2-standard-16", "memory_GB":64} | hardware | json descriptor of target hardware environment |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| execution\_config | {“number”: 1, “repeat”: 10, “min\_repeat\_ms”, 0} | statistics | workload execution parameters |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| metrics | {“accuracy”: 48.5,“compilation_ms_mean”: 12} | statistics | other metrics |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-| runtime_raw | [{"runtime_ms": 12, ...}, {"runtime_ms":13,...},...] | statistics | optional raw metrics array |
-+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+------------------------------------------------------------------------------+
-
-
-
-Storage format
-~~~~~~~~~~~~~~
-Currently we're prototyping benchmark data as JSON objects for extensibility and convenience, especially in early versions of the schema. However, as we scale up benchmark aggregation and stabilize parameters, we anticipate switching to a columnar format, such as Arrow or Parquet.
-
-Here is sample data encoded as JSON:
-
-::
-
- {
- "workload":"arcface_resnet100",
- "engine":"tvm",
- "hardware":"gcp-c2-standard-16",
- "runtime_ms_mean":109.43004820081924,
- "runtime_ms_std":0.09078385126800587,
- "timestamp":"20191123003411",
- "schema_version":"0.1",
- "metadata":{
- "docker_tag":"tlcpack/ci-gpu:v0.53"
- },
- "workload_args":{
- "input_shape_dict":{
- "data":[
- 1,
- 3,
- 112,
- 112
- ]
- },
- "input_type_dict":{
- "data":"float32"
- },
- "input_value_dict":{}
- },
- "workload_metadata":{
- "class":"vision",
- "doc_url":"https://github.com/onnx/models/blob/main/vision/body_analysis/arcface/README.md",
- "md5":"66074b860f905295aab5a842be57f37d",
- "opset":8,
- "type":"body_analysis",
- "url":"https://s3.amazonaws.com/onnx-model-zoo/arcface/resnet100/resnet100.tar.gz"
- },
- "engine_version":"1.0.0",
- "engine_config":{},
- "compilation_config":{
- "relay_opt_level": 3
- },
- "software_config":{
- "os":"ubuntu:18.04",
- "pip":{
- "docker":"4.1.0",
- "gitpython":"3.0.4",
- "numpy":"1.17.4",
- "onnx":"1.6.0"
- }
- },
- "runtime_config":{},
- "hardware_config":{
- "cloud_machine_type":"c2-standard-16",
- "cloud_provider":"GCP",
- "cpu_count":16,
- "cpu_platform":"Intel Cascade Lake",
- "memory_GB":64
- },
- "execution_config":{},
- "metrics":{}
- }
diff --git a/docs/_sources/arch/convert_layout.rst.txt b/docs/_sources/arch/convert_layout.rst.txt
deleted file mode 100644
index 51917fce44..0000000000
--- a/docs/_sources/arch/convert_layout.rst.txt
+++ /dev/null
@@ -1,269 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-.. http://www.apache.org/licenses/LICENSE-2.0
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-===================
-Convert Layout Pass
-===================
-**Author**: `Animesh Jain <https://github.com/anijain2305>`_
-
-*************
-1. Background
-*************
-
-Data layout format describes how the data is laid out in the memory. For example, Tensorflow framework default data layout for convolution operator is NHWC, i.e, the data is 4-dimensions and is laid out in row-major format with N being the first dimension and C being the last dimension. Data layout has a major role in model performance, significantly affecting spatial and temporal locality. For example, Intel x86 backend in TVM prefers layout as NCHWc where the C dimension is tiled in 2 [...]
-
-Essentially, TVM has to deal with data layouts throughout the compiler toolchain - Framework parsers, Relay layout transformations, and TOPI schedules. As we move towards third-party codegen integration, which might have their own data layout restrictions, handling layouts at all levels in TVM toolchain is going to become even more challenging. Therefore, we developed a new Relay pass - **ConvertLayout** -- to reduce some of the complications that arise due to layout handling.
-
-If you directly want to understand the usage of ConvertLayout Pass, directly jump to Section 4 - Usage.
-
-**************************
-2. Motivation and Overview
-**************************
-
-Let's look at a simple scenario to understand the complications that arise due to different layouts - Suppose we want to compile a Tensorflow NHWC graph for an ARM edge device. But, suppose we currently support only NCHW schedules in TOPI for ARM. So, there is a mismatch between framework layout and TOPI-supported layout. One way to deal with this mismatch is to insert layout transforms before each and after convolution, such that resulting convolution has NCHW input data layout and can [...]
-
-We encountered similar problems in other use cases as well
-
-- No way to run TFLite graphs on Nvidia GPUs. TOPI has NCHW-only schedules for GPUs.
-- Ever-complicating logic in AlterOpLayout for convolution to support different pairs of layout transformations.
-- Sub-optimal performance for TF graphs due to extra layout transforms.
-- Complication in third-party codegen integrations like TensorRT that prefers data layout to be in one format.
-
-To solve these problems, we introduced *ConvertLayout* pass that sets up the infrastructure to change the data layout of the whole graph with minimal number of data layout transforms. In ideal cases, we will have only 2 layout transforms for data, one at the start and one at the end. An example to show the transformation is below
-
-
-.. code-block:: python
-
- # Original graph - 2 convolutions in NHWC format.
- fn (%x: Tensor[(1, 56, 56, 64), float32], %weight1: Tensor[(3, 3, 64, 32), float32], %weight2: Tensor[(3, 3, 32, 32), float32]) {
- %0 = nn.conv2d(%x, %weight1, padding=[1, 1], channels=32, kernel_size=[3, 3], data_layout="NHWC", kernel_layout="HWIO");
- %1 = nn.relu(%0);
- %2 = nn.conv2d(%1, %weight2, padding=[1, 1], channels=32, kernel_size=[3, 3], data_layout="NHWC", kernel_layout="HWIO");
- nn.relu(%2)
- }
-
- # After ConvertLayout - For data, there is a transform at the start and at the end.
- # For weights, there are transforms to adapt to NCHW layout. These will be removed by FoldConstant pass.
- fn (%x: Tensor[(1, 56, 56, 64), float32], %weight1: Tensor[(3, 3, 64, 32), float32], %weight2: Tensor[(3, 3, 32, 32), float32]) {
- %0 = layout_transform(%x, src_layout="NHWC", dst_layout="NCHW") /* ty=Tensor[(1, 64, 56, 56), float32] */;
- %1 = layout_transform(%weight1, src_layout="HWIO", dst_layout="OIHW") /* ty=Tensor[(32, 64, 3, 3), float32] */;
- %2 = nn.conv2d(%0, %1, padding=[1, 1], channels=32, kernel_size=[3, 3]) /* ty=Tensor[(1, 32, 56, 56), float32] */;
- %3 = nn.relu(%2) /* ty=Tensor[(1, 32, 56, 56), float32] */;
- %4 = layout_transform(%weight2, src_layout="HWIO", dst_layout="OIHW") /* ty=Tensor[(32, 32, 3, 3), float32] */;
- %5 = nn.conv2d(%3, %4, padding=[1, 1], channels=32, kernel_size=[3, 3]) /* ty=Tensor[(1, 32, 56, 56), float32] */;
- %6 = nn.relu(%5) /* ty=Tensor[(1, 32, 56, 56), float32] */;
- layout_transform(%6, src_layout="NCHW", dst_layout="NHWC") /* ty=Tensor[(1, 56, 56, 32), float32] */
- }
-
-
-*********
-3. Design
-*********
-
-Before delving into ConvertLayout pass, let's categorize the operators into 3 categories based on their sensitivity to data layouts. This categorization will be useful later to understand Convertlayout pass details.
-
-- **Layout agnostic** - Relu, pow etc. These operators are not affected, neither functionality nor performance, by data layouts.
-- **Lightly-layout sensitive** - pad, concatenate, reduce ops like sum etc. These operators have some attributes that are functionally affected if we do a layout transformation before them. However, performance-wise, the difference is not significant. For these operators, it is beneficial to just adapt to the previous operator output data layout.
-- **Heavily-layout sensitive** - Convolution, conv2d_transpose etc. These operators are heavily affected, both functionally and performance-wise, by data layouts. They also have data layout as the op attribute. Typically, it is beneficial to modify the input data layouts for these operators (if its not a performant data layout), while the rest of *layout agnostic* and *lightly-layout sensitive* operators adapt to the layout governed by the output of these *heavliy-layout sensitive* operators.
-
-
-Let us now look at two relevant Relay operator properties. Each relay operator has properties, like InferType, that can be defined by a TVM developer. Typically, a Relay pass traverses the graph operator-by-operator and reads these operator properties. For example, InferType pass looks at the InferType property of on operator, determines its output shape and type, and then passes it to the next operator InferType property. Similarly, in our context, we have 2 such properties - *FTVMConve [...]
-
-- Run FTVMConvertLayout property - This allows the developers to transform the original Relay expr into a new Relay expr with new layouts, allowing user-defined layout alteration. There is a python callback for developer's ease. This is used only for heavily-layout sensitive operators.
-- Run FTVMInferCorretLayout property - We can view this as layout inference. It looks at the original input layout and the new input layouts, which are either coming from previous operator or from the FTVMConvertLayout modified expr (if it was used). This can be used by lightly-layout sensitive operators to adapt its attributes to new data layouts. Layout inference happens for each operator.
-- Automatic insertion of layout transforms - The previous step - layout inference - sets the new layout for the input exprs. If these layouts are different from the original layouts, then this component automatically inserts a layout transform. Therefore, a developer does not need to do anything for this component.
-
-These steps happen for each operator in sequence, where ConvertLayout pass keeps on passing the new layouts to the next operator properties, finally resulting in modifying the whole graph operator-by-operator. Now, let's look at a couple of examples of how to define the two properties.
-
-**FTVMConvertLayout - Python callback for layout alteration** - This is used for *heavily-layout sensitive* operators. For example, one can return a new convolution operator with new data and kernel layout. The other 2 components will infer layout and insert layout transforms if needed. One example for convolution operator is as follows where we are converting to NCHW layout.
-
-.. code-block:: python
-
- @reg.register_convert_op_layout("nn.conv2d")
- def convert_conv2d(attrs, inputs, tinfos, desired_layouts):
- """Convert Layout pass registration for conv2d op.
-
- Parameters
- ----------
- attrs : tvm.attrs.Attrs
- Attributes of current convolution
- inputs : list of tvm.relay.Expr
- The args of the Relay expr to be legalized
- tinfos : list of types
- List of input and output types
- desired_layouts : list of layout strings
- List of layouts defining our desired
- layout for the data and kernel inputs respectively.
-
- Returns
- -------
- result : tvm.relay.Expr
- The transformed expr
- """
-
- from tvm import relay
- data, weight = inputs
- new_attrs = dict(attrs)
-
- # We expect 2 desired layouts to be specified, one for the data and one for the kernel.
- assert len(desired_layouts) == 2, "A desired layout is expected for both of nn.conv2d's inputs"
-
- # Use the first entry in desired layouts which specifies the data layout.
- # The expected ordering of layouts for this operator is defined by this function.
- desired_data_layout, desired_kernel_layout = map(str, desired_layouts)
-
- assert desired_data_layout != "default", "Data layout cannot be default"
-
- new_attrs['data_layout'] = desired_data_layout
-
- if desired_data_layout == 'NCHW':
- if desired_kernel_layout != 'default':
- new_attrs['kernel_layout'] = desired_kernel_layout
- else:
- new_attrs['kernel_layout'] = 'OIHW'
- # Actual insertion of layout transforms is taken care internally
- # by ConvertLayout pass.
- return relay.nn.conv2d(data, weight, **new_attrs)
-
- raise ValueError('Layout %s is not yet supported' % desired_data_layout)
-
-
-**FInferCorrectLayout - Layout inference** - Currently, this attribute is exposed only in C++. This function takes original input layouts and the new input layouts (passed from the previous operator or from the python callback for layout alteration), and infers the final data layouts. Layout inference is called for each operator. The usage might vary for different operator categories. For layout agnostic operators, we just want to return the new data layouts in this function. For lightly [...]
-
-First example is for layout agnostic operators. These operators do not have any operator attributes that are affected by data layouts, so we just adapt to new layouts.
-
-.. code-block:: c++
-
- // For operator set its attributes like following
- // .set_attr<FInferCorrectLayout>("FInferCorrectLayout", ElemwiseArbitraryLayout);
-
- // Take arbitrary input layouts and copy to outputs.
- inline Array<Array<Layout>> ElemwiseArbitraryLayout(const Attrs& attrs,
- const Array<Layout>& new_in_layouts,
- const Array<Layout>& old_in_layouts,
- const Array<Array<IndexExpr>> &old_in_shapes) {
- Layout ret;
-
- if (new_in_layouts.defined()) {
- ICHECK_GE(new_in_layouts.size(), 1);
- ret = new_in_layouts[0];
- } else {
- for (size_t i = 0; i < old_in_layouts.size(); ++i) {
- if (old_in_layouts[i].defined()) {
- ret = old_in_layouts[i];
- break;
- }
- }
- }
-
- return Array<Array<Layout>>{Array<Layout>(old_in_layouts.size(), ret), {ret}};
- }
-
-
-Second example is for a lightly-layout sensitive operator - batch normalization. BatchNorm has an axis operator that has to change when we go from NHWC to NCHW data layout. (Similar handling also needs to be for heavily-layout sensitive operators)
-
-
-.. code-block:: c++
-
- Array<Array<Layout>> BatchNormInferCorrectLayout(const Attrs& attrs,
- const Array<Layout>& new_in_layouts,
- const Array<Layout>& old_in_layouts,
- const Array<Array<IndexExpr>>& old_in_shapes) {
- BatchNormAttrs* param = const_cast<BatchNormAttrs*>(attrs.as<BatchNormAttrs>());
-
- size_t axis =
- param->axis < 0 ? param->axis + old_in_shapes[0].size() : static_cast<size_t>(param->axis);
-
- Layout ret = Layout::Undef();
-
- // For example, consider old_layout = NHWC, and new_layout = NCHW, and param->axis = 3
-
- if (new_in_layouts.defined() && old_in_layouts.defined()) {
- // Get the new C axis. Extract the dim in old layout. Find the index of that dim in next layout.
-
- // Following line gives bn_dim = C as old_layout = NHWC, axis = 3
- const auto& bn_dim = old_in_layouts[0][axis];
-
- // The new_index is 1 because new_layout = NCHW and bn_dim is C
- auto new_index = new_in_layouts[0].IndexOf(bn_dim);
-
- // We modify the layout-dependent attribute here - axis to 1.
- param->axis = new_index;
-
- // Finally, we adapt to the new layout.
- ret = new_in_layouts[0];
-
- } else if (old_in_layouts.defined()) {
- ret = old_in_layouts[0];
- }
-
- // In case both new and old layouts are undefined, then there is no need of a change.
- // ConvertLayout pass skips the automatic insertion of layout transforms in this case.
-
- // Following line is not important to tutorial. But, layout inference needs to define
- // the layout for all input and output data layouts. For batch norm, the other inputs
- // and outputs are vector having length of C dim in the input. So, we set the other
- // layouts as C. BN has 5 inputs, 3 outputs. The last 4 inputs and last 2 outputs
- // have "C" layout.
- Layout c_layout = Layout("C");
-
- return Array<Array<Layout>>{{ret, c_layout, c_layout, c_layout, c_layout},
- {ret, c_layout, c_layout}};
- }
-
-
-********
-4. Usage
-********
-.. _convert-layout-usage:
-
-ConvertLayout pass is extremely easy to use. The pass is not a part of default relay.build pipeline. The intended usage is to call it between the framework-to-relay parser and relay.build module call.
-
-In order to specify the layouts to convert to, we create a mapping of heavily-layout sensitive operators to a list of the desired layouts for that operator. The first example below specifies data layout, we allow the kernel layout to be automatically converted to one that is supported by TVM (for that particular data layout and operator). This is specified by the use of the "default" keyword. The second example shows how we could have also converted to a specific kernel layout of our cho [...]
-
-.. code-block:: python
-
- # TFlite framework to Relay parser - Default layout is NHWC
- mod, params = relay.frontend.from_tflite(tflite_model,
- shape_dict=shape_dict,
- dtype_dict=dtype_dict)
-
- # We assume our model's heavily-layout sensitive operators only consist of nn.conv2d
- desired_layouts = {'nn.conv2d': ['NCHW', 'default']}
-
- # Convert the layout to NCHW
- # RemoveUnunsedFunctions is used to clean up the graph.
- seq = tvm.transform.Sequential([relay.transform.RemoveUnusedFunctions(),
- relay.transform.ConvertLayout(desired_layouts)])
- with tvm.transform.PassContext(opt_level=3):
- mod = seq(mod)
-
- # Call relay compilation
- with relay.build_config(opt_level=3):
- graph, lib, params = relay.build(mod, target, params=params)
-
-
-.. code-block:: python
-
- desired_layouts = {'nn.conv2d': ['NCHW', 'OIHW']}
- pass = relay.transform.ConvertLayout(desired_layouts)
-
-
-The ordering of the layouts is defined by the implementation of `register_convert_op_layout("OPNAME")`, you can refer to the docstring which should explicitly state the expected layout. In the examples above it's [data_layout, kernel_layout].
-
-Current implementation has support for almost all the operators commonly used in image classification models. However, if one encounters too many data layout transforms in the graph, it is highly likely that there is an operator whose layouts need special handling as described in Section 3. Some pull requests that can help in such a situation are
-
-- Layout inference for `Batch Norm <https://github.com/apache/tvm/pull/4600>`_ - Batch normalization falls into the category of lightly-sensitive operator. The PR shows how to handle the layout inference for batch norm.
-- Python Callback for `Convolution <https://github.com/apache/tvm/pull/4335>`_- For highly-sensitive operators, one might have to do python callback as well. The PR shows how to define a python callback function for Convolution operator.
diff --git a/docs/_sources/arch/frontend/tensorflow.rst.txt b/docs/_sources/arch/frontend/tensorflow.rst.txt
deleted file mode 100644
index dde7179d90..0000000000
--- a/docs/_sources/arch/frontend/tensorflow.rst.txt
+++ /dev/null
@@ -1,254 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-TensorFlow Frontend
-===================
-
-The TensorFlow frontend helps in importing TensorFlow models into TVM.
-
-Supported versions:
-
-- 1.12 and below
-
-Tested models:
-
-- Inception (V1/V2/V3/V4)
-- Resnet (All)
-- Mobilenet (V1/V2 All)
-- Vgg (16/19)
-- BERT (Base/3-layer)
-
-Preparing a Model for Inference
--------------------------------
-
-Remove Unneeded Nodes
-~~~~~~~~~~~~~~~~~~~~~
-
-The export process will remove many nodes that are not needed for inference, but unfortunately will leave some remaining. The nodes that should be manually removed are:
-
-- Dropout, including `Dropout`_ and `DropoutWrapper`_
-- `Assert`_
-
-.. _Dropout: https://www.tensorflow.org/api_docs/python/tf/nn/dropout
-.. _DropoutWrapper: https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/nn/rnn_cell/DropoutWrapper?hl=hr
-.. _Assert: https://www.tensorflow.org/api_docs/python/tf/debugging/Assert
-
-Convert None Dimensions to Constants
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-TVM has minimal support for dynamic tensor shapes. Dimensions that are ``None`` should be replaced with constants. For example, a model may accept an input with shape ``(None,20)``. This should be converted to a shape like ``(1,20)``. The model should be modified accordingly to ensure that these shapes match throughout the graph.
-
-Export
-~~~~~~
-
-TensorFlow frontend expects a frozen protobuf (.pb) or saved model as input. It currently does not support checkpoint (.ckpt). The graphdef needed by the TensorFlow frontend can be extracted from the active session, or by using the `TFParser`_ helper class.
-
-.. _TFParser: https://github.com/apache/tvm/blob/main/python/tvm/relay/frontend/tensorflow_parser.py
-
-The model should be exported with a number of transformations to prepare the model for inference. It is also important to set ```add_shapes=True```, as this will embed the output shapes of each node into the graph. Here is one function to export a model as a protobuf given a session:
-
-.. code:: python
-
- import tensorflow as tf
- from tensorflow.tools.graph_transforms import TransformGraph
-
- def export_pb(session):
- with tf.gfile.GFile("myexportedmodel.pb", "wb") as f:
- inputs = ["myinput1", "myinput2"] # replace with your input names
- outputs = ["myoutput1"] # replace with your output names
- graph_def = session.graph.as_graph_def(add_shapes=True)
- graph_def = tf.graph.util.convert_variables_to_constants(session, graph_def, outputs)
- graph_def = TransformGraph(
- graph_def,
- inputs,
- outputs,
- [
- "remove_nodes(op=Identity, op=CheckNumerics, op=StopGradient)",
- "sort_by_execution_order", # sort by execution order after each transform to ensure correct node ordering
- "remove_attribute(attribute_name=_XlaSeparateCompiledGradients)",
- "remove_attribute(attribute_name=_XlaCompile)",
- "remove_attribute(attribute_name=_XlaScope)",
- "sort_by_execution_order",
- "remove_device",
- "sort_by_execution_order",
- "fold_batch_norms",
- "sort_by_execution_order",
- "fold_old_batch_norms",
- "sort_by_execution_order"
- ]
- )
- f.write(graph_def.SerializeToString())
-
-Another method is to `export and freeze the graph <https://github.com/tensorflow/models/tree/master/research/slim#exporting-the-inference-graph>`_.
-
-Import the Model
-----------------
-
-Explicit Shape:
-~~~~~~~~~~~~~~~
-
-To ensure shapes can be known throughout the entire graph, pass the ```shape``` argument to ```from_tensorflow```. This dictionary maps input names to input shapes. Please refer to these `test cases <https://github.com/apache/tvm/blob/main/tests/python/frontend/tensorflow/test_forward.py#L36>`_ for examples.
-
-Data Layout
-~~~~~~~~~~~
-
-Most TensorFlow models are released with NHWC layout. NCHW layout often provides better performance, especially on GPU. The TensorFlow frontend can automatically convert the model's data layout by passing the argument ```layout='NCHW'``` to ```from_tensorflow```.
-
-Best Practices
---------------
-
-- Use static tensor shapes instead of dynamic shapes (remove ```None``` dimensions).
-- Use static RNN instead of dynamic RNN, as ```TensorArray``` isn't supported yet.
-
-Supported Ops
--------------
-
-- Abs
-- Add
-- AddN
-- All
-- Any
-- ArgMax
-- ArgMin
-- AvgPool
-- BatchMatMul
-- BatchMatMulV2
-- BatchNormWithGlobalNormalization
-- BatchToSpaceND
-- BiasAdd
-- BroadcastTo
-- Cast
-- Ceil
-- CheckNumerics
-- ClipByValue
-- Concat
-- ConcatV2
-- Conv2D
-- Cos
-- Tan
-- CropAndResize
-- DecodeJpeg
-- DepthwiseConv2dNative
-- DepthToSpace
-- Dilation2D
-- Equal
-- Elu
-- Enter
-- Erf
-- Exit
-- Exp
-- ExpandDims
-- Fill
-- Floor
-- FloorDiv
-- FloorMod
-- FusedBatchNorm
-- FusedBatchNormV2
-- Gather
-- GatherNd
-- GatherV2
-- Greater
-- GreaterEqual
-- Identity
-- IsFinite
-- IsInf
-- IsNan
-- LeakyRelu
-- LeftShift
-- Less
-- LessEqual
-- Log
-- Log1p
-- LoopCond
-- LogicalAnd
-- LogicalOr
-- LogicalNot
-- LogSoftmax
-- LRN
-- LSTMBlockCell
-- MatMul
-- Max
-- MaxPool
-- Maximum
-- Mean
-- Merge
-- Min
-- Minimum
-- MirrorPad
-- Mod
-- Mul
-- Neg
-- NextIteration
-- NotEqual
-- OneHot
-- Pack
-- Pad
-- PadV2
-- Pow
-- Prod
-- Range
-- Rank
-- RealDiv
-- Relu
-- Relu6
-- Reshape
-- ResizeBilinear
-- ResizeBicubic
-- ResizeNearestNeighbor
-- ReverseV2
-- RightShift
-- Round
-- Rsqrt
-- Select
-- Selu
-- Shape
-- Sigmoid
-- Sign
-- Sin
-- Size
-- Slice
-- Softmax
-- Softplus
-- SpaceToBatchND
-- SpaceToDepth,
-- Split
-- SplitV
-- Sqrt
-- Square
-- SquareDifference
-- Squeeze
-- StridedSlice
-- Sub
-- Sum
-- Switch
-- Tanh
-- TensorArrayV3
-- TensorArrayScatterV3
-- TensorArrayGatherV3
-- TensorArraySizeV3
-- TensorArrayWriteV3
-- TensorArrayReadV3
-- TensorArraySplitV3
-- TensorArrayConcatV3
-- Tile
-- TopKV2
-- Transpose
-- TruncateMod
-- Unpack
-- UnravelIndex
-- Where
-- ZerosLike
diff --git a/docs/_sources/arch/hybrid_script.rst.txt b/docs/_sources/arch/hybrid_script.rst.txt
deleted file mode 100644
index a4fce342f7..0000000000
--- a/docs/_sources/arch/hybrid_script.rst.txt
+++ /dev/null
@@ -1,100 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-Hybrid Frontend Developer Guide
-===============================
-
-If you are a developer:
-
-1. who is trying writing some preliminary patterns that have not been supported by TVM yet,
-maybe :ref:`hybrid-langref-label` is a better place for you.
-
-2. who wants to know the implementation details of this module, you are right here!
-
-Features
---------
-
-Software Emulation
-~~~~~~~~~~~~~~~~~~
-
-In software emulation, the most interesting thing is the decorator ``tvm.te.hybrid.script``.
-This decorator helps 2 things:
-
-1. Importing runtime variables
-
-2. Overloading the function according to the arguments passed
-
-Correct me if I am wrong: I believe that how 1. is implemented is dangerous, but I have no
-choice. What I did is to add those names into python dict ``func.__global__`` and after
-the call to ``func`` is done, those names will be cleaned up.
-
-Overload is simple: the decorator checks the arguments' types and determines which function
-should be actually called.
-
-
-Backend Compilation
-~~~~~~~~~~~~~~~~~~~
-
-Compilation is a large module, you can see ``python/tvm/te/hybrid/`` for more
-details. The first stage determines the usage, or more accurately the
-declaration of each variable and the second stage does the actual IR
-generation.
-
-Attributes
-~~~~~~~~~~
-
-So far, ONLY tensors' `shape` attribute is supported. You can see ``visit_Subscript``
-in ``python/tvm/te/hybrid/parser.py`` for more details. This is a hacky solution, I just
-check the attributes when subscript.
-
-Loops
-~~~~~
-
-In HalideIR, loops have in total 4 types: ``serial``, ``unrolled``, ``parallel``, and ``vectorized``.
-
-
-.. note::
-
- Unlike what that is in HalideIR, in ``loop_type(a, b)``, ``a`` is the starting point and ``b``
- is the trip count of iterations. Here ``loop_type(a, b)`` indicates ``[a, b)``. Thus, when lowering it
- to HalideIR, we need to do ``start, extent = a, b - a``
-
-
-.. note::
-
- In HalideIR those are enums, they are in passive form.
- Here we use active form to annotate loops, because they are ready to run.
-
-
-Variables
-~~~~~~~~~
-
-Because there is no variables in ``HalideIR``, all the mutable variables will be lowered to an array with size 1.
-It takes the first store of a variable as its declaration.
-
-Math Intrinsics
-~~~~~~~~~~~~~~~
-So far, these math intrinsics, ``log``, ``exp``, ``sigmoid``, ``tanh``, ``power``, and ``popcount``, are supported.
-Math intrinsics will be imported by the decorator. Most of the intrinsics are borrowed by library implementation
-except ``popcount`` and ``sigmoid``. I implemented them manually.
-
-
-Casting
-~~~~~~~
-
-You can cast values by using the keywords ``uint8``, ``uint16`` ``uint32``, ``uint64``, ``int8``, ``int16``, ``int32``, ``int64``,
-``float16``, ``float32``, ``float64``.
diff --git a/docs/_sources/arch/index.rst.txt b/docs/_sources/arch/index.rst.txt
index 17884a7742..cf4829268e 100644
--- a/docs/_sources/arch/index.rst.txt
+++ b/docs/_sources/arch/index.rst.txt
@@ -18,46 +18,37 @@
Design and Architecture
=======================
-This document is intended for developers who want to understand the
-architecture of TVM and/or actively develop on the project.
+This document is intended for developers who want to understand the architecture of Apache TVM and/or actively develop on the project.
This page is organized as follows:
-- The `Example Compilation Flow`_ gives an overview of the steps that TVM takes to turn a high level description of a model into a deployable module.
+- The `Overall Flow`_ gives an overview of the steps that TVM takes to turn a high level description of a model into a deployable module.
To get started, please read this section first.
-
-- The `Logical Architecture Components`_ section describes the logical components.
- The sections after are specific guides focused on each logical component, organized
- by the component's name.
-
-- The :ref:`Device/Target Interactions <tvm-target-specific-overview>`
- page describes how TVM interacts with each supported physical device
- and code-generation target.
-
-- Feel free to also check out the :ref:`dev-how-to` for useful development tips.
+- Brief introduction to the key components of the TVM stack. Feel free to also check out the :ref:`TensorIR Deep Dive <tensor-ir-deep-dive>`
+ and :ref:`Relax Deep Dive <relax-deep-dive>` for more details about the two major components in the TVM stack.
This guide provides a few complementary views of the architecture.
First, we review a single end-to-end compilation flow and discuss the key data structures and the transformations.
This runtime-based view focuses on the interactions of each components when running the compiler.
Then we will review the logical modules of the codebase and their relationship. This part provides a static overarching view of the design.
-
-Example Compilation Flow
-------------------------
+Overall Flow
+------------
In this guide, we will study an example compilation flow in the compiler. The figure below shows the flow. At a high-level, it contains several steps:
-- Import: The frontend component ingests a model into an IRModule, which contains a collection of functions that internally represent the model.
-- Transformation: The compiler transforms an IRModule to another functionally equivalent or approximately
+- **Model Creation**: Create the IRModule to be optimized and compiled, which contains a collection of functions that internally represent the model.
+ Users can manually construct IRModule via NNModule, TVMScript, or import a pre-trained model from from Relax frontend.
+- **Transformation**: The compiler transforms an IRModule to another functionally equivalent or approximately
equivalent(e.g. in the case of quantization) IRModule. Many of the transformations are target (backend) independent.
We also allow target to affect the configuration of the transformation pipeline.
-- Target Translation: The compiler translates(codegen) the IRModule to an executable format specified by the target.
+- **Target Translation**: The compiler translates(codegen) the IRModule to an executable format specified by the target.
The target translation result is encapsulated as a `runtime.Module` that can be exported, loaded, and executed on the target runtime environment.
-- Runtime Execution: the user loads back a `runtime.Module` and runs the compiled functions in the supported runtime environment.
+- **Runtime Execution**: the user loads back a `runtime.Module` and runs the compiled functions in the supported runtime environment.
-.. figure:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_dyn_workflow.svg
+.. figure:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_overall_flow.svg
:align: center
- :width: 85%
+ :width: 80%
Key data structures
@@ -70,13 +61,14 @@ components that either define a collection of key data structures or transformat
**IRModule** is the primary data structure used across the entire stack. An IRModule (intermediate representation module)
contains a collection of functions. Currently, we support two primary variants of functions.
-- **relay::Function** is a high-level functional program representation. A relay.Function usually corresponds to an end-to-end model.
- You can view a relay.Function as a computational graph with additional support for control-flow, recursion, and complex data structures.
+- **relax::Function** is a high-level functional program representation. A relax.Function represents high-level graph structure,
+ usually corresponds to an end-to-end model or a sub-graph of the overall model. You can view a relax.Function as a computational
+ graph with additional support for control-flow, and complex data structures.
- **tir::PrimFunc** is a low-level program representation that contains elements including loop-nest choices, multi-dimensional load/store,
threading, and vector/tensor instructions. It is usually used to represent an operator program that executes a (possibly-fused) layer in a model.
-During the compilation, a relay function may be lowered to multiple tir::PrimFunc functions and a top-level function that calls into
-those tir::PrimFunc functions.
+During the compilation and transformation, all relax operators are lowered to ``tir::PrimFunc`` or ``TVM PackedFunc``, which can be executed directly
+on the target device, while the calls to relax operators are lowered to calls to low-level functions (e.g. ``R.call_tir`` or ``R.call_dps``).
Transformations
~~~~~~~~~~~~~~~
@@ -86,44 +78,35 @@ Now that we have covered the key data structures, let us talk about the transfor
- optimization: transform a program to an equivalent, possibly more optimized version.
- lowering: transform a program to a lower-level representation that is closer to the target.
-**relay/transform** contains a collection of passes that optimize the model. The optimizations include common program
-optimizations such as constant folding and dead-code elimination, and tensor-computation specific passes such as layout
-transformation and scaling factor folding.
-
-Near the end of the relay optimization pipeline, we will run a pass(FuseOps) to break the end-to-end function(e.g. MobileNet)
-into sub-function(e.g. conv2d-relu) segments. We call these segments of functions.
-This process helps us to divide the original problem into two sub-problems:
-
-- Compilation and optimization for each sub-function.
-- Overall execution structure: we need to do a sequence of calls into the generated sub-functions to execute the whole model.
-
-We use the low-level tir phase to compile and optimize each sub-functions. For specific targets, we may also directly go to the target translation
-phase and use external code generators.
-
-There are a few different ways(in relay/backend) to handle the calls into the overall execution problem. For simple models with known shapes and no control flow, we can lower to a graph executor that stores the execution structure in a graph. We also support a virtual machine backend for dynamic executions. Finally, we plan to support ahead of time compilation that compiles the high-level execution structure into the executable and generated primitive functions. All of these execution mo [...]
+relax transformations
+^^^^^^^^^^^^^^^^^^^^^
+relax transformations contain a collection of passes that apply to relax functions. The optimizations include common graph-level
+optimizations such as constant folding and dead-code elimination for operators, and backend-specific optimizations such as library dispatch.
-**tir/transform** contains transformation passes for TIR level functions. Many tir passes serve the purpose of lowering. For example, there are passes to flatten multi-dimensional access to one-dimensional pointer access, to expand the intrinsics into target-specific ones, and to decorate the function entry to meet the runtime calling convention. Of course, there are also optimizations passes, such as access index simplification and dead code elimination.
+tir transformations
+^^^^^^^^^^^^^^^^^^^
+tir transformations contain a collection of passes that apply to tir functions. There are two major types of transformations:
-Many low-level optimizations can be handled in the target phase by the LLVM, CUDA C, and other target compilers. As a result, we leave low-level optimizations such as register allocation to the downstream compilers and only focus on optimizations that are not covered by them.
+- **TensorIR schedule**: TensorIR schedules are designed to optimize the TensorIR functions for a specific target, with user-guided instructions and control how the target code is generated.
+ For CPU targets, TIR PrimFunc can generate valid code and execute on the target device without schedule but with very-low performance. However, for GPU targets, the schedule is essential
+ for generating valid code with thread bindings. For more details, please refer to the :ref:`TensorIR Transformation <tir-transform>` section. Additionally, we provides ``MetaSchedule`` to
+ automate the search of TensorIR schedule.
+- **Lowering Passes**: These passes usually perform after the schedule is applied, transforming a TIR PrimFunc into another functionally equivalent PrimFunc, but closer to the
+ target-specific representation. For example, there are passes to flatten multi-dimensional access to one-dimensional pointer access, to expand the intrinsics into target-specific ones,
+ and to decorate the function entry to meet the runtime calling convention.
-Search-space and Learning-based Transformations
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Many low-level optimizations can be handled in the target phase by the LLVM, CUDA C, and other target compilers. As a result, we leave low-level optimizations such as register allocation
+ to the downstream compilers and only focus on optimizations that are not covered by them.
-The transformation passes we described so far are deterministic and rule-based. One design goal of the TVM stack is to support high-performance code optimizations for different hardware platforms. To do so, we will need to investigate as many optimization choices as possible, including but not limited to, multi-dimensional tensor access, loop tiling behavior, special accelerator memory hierarchy, and threading.
+cross-level transformations
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Apache TVM brings a unity strategy to optimize the end-to-end models. As the IRModule includes both relax and tir functions, the cross-level transformations are designed to mutate
+the IRModule by applying different transformations to these two types of functions.
-It is hard to define a heuristic to make all of the choices. Instead, we will take a search and learning-based approach.
-We first define a collection of actions we can take to transform a program. Example actions include loop transformations, inlining,
-vectorization. We call these actions **scheduling primitives**. The collection of scheduling primitives defines a search space of possible
-optimizations we can make to a program. The system then searches over different possible scheduling
-sequence to pick the best scheduling combination.
-The search procedure is usually guided by a machine learning algorithm.
-
-We can record the best schedule sequence for an (possibly-fused) operator once the search is completed. The compiler can then just lookup the best
-schedule sequence and apply it to the program. Notably, this schedule application phase is **exactly like** the rule-based transformations,
-enabling us to share the same interface convention with tradition passes.
-
-We use search based optimizations to handle the initial tir function generation problem. This part of the module is called AutoTVM(auto_scheduler).
-We expect to expand the learning-based transformations to more areas as we continue to develop the TVM stack.
+For example, ``relax.LegalizeOps`` pass mutates the IRModule by lowering relax operators, add corresponding TIR PrimFunc into the IRModule, and replace the relax operators
+with calls to the lowered TIR PrimFunc. Another example is operator fusion pipeline in relax (including ``relax.FuseOps`` and ``relax.FuseTIR``), which fuse multiple consecutive tensor operations
+into one. Different from the previous implementations, relax fusion pipeline analyzes the pattern of TIR functions and detects the best fusion rules automatically rather
+than human-defined operator fusion patterns.
Target Translation
~~~~~~~~~~~~~~~~~~
@@ -204,19 +187,6 @@ except that the data structure of interest changes from the numpy.ndarray to tvm
- Manipulate the IR directly using TVM's python API.
-Logical Architecture Components
--------------------------------
-
-.. figure:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_static_overview.svg
- :align: center
- :width: 85%
-
- TVM Architecture Diagram
-
-The above figure shows the major logical components in the project. Please read the following sections
-for information about the components and their relations.
-
-
tvm/support
-----------
The support module contains the most common utilities for the infrastructure, such as generic arena allocator, socket, and logging.
@@ -243,22 +213,19 @@ These hardware-specific runtime modules define APIs for device memory allocation
device and benchmark the execution performance. The rpc infrastructure enables data collection from a wide range of hardware backends
for learning-based optimizations.
-
.. toctree::
:maxdepth: 1
runtime
-
.. toctree::
:maxdepth: 1
debugger
- virtual_machine
introduction_to_module_serialization
device_target_interactions
-
+.. TODO(tvm-team) add a section about relax vm here
tvm/node
--------
@@ -275,11 +242,9 @@ Thanks to the node module, we can directly access any field of the TVM's IRNode
# we can directly use the field name to access the IR structures
assert y.a == x
-
We can also serialize arbitrary IR node into a JSON format, and load them back.
The ability to save/store, and inspect an IR node provides a foundation for making the compiler more accessible.
-
tvm/ir
------
The `tvm/ir` folder contains the unified data structure and interfaces across for all IR function variants.
@@ -331,11 +296,25 @@ in the target and builtin information registered to each target id(cuda, opencl)
device_target_interactions
+tvm/relax
+---------
+
+Relax is the high-level IR used to represent the computational graph of a model. Various optimizations are defined in ``relax.transform``.
+Note that Relax usually works closely the the TensorIR IRModule, most of the transformations are applied on the both Relax and TensorIR functions
+in the IRModule. Please refer to the :ref:`Relax Deep Dive <relax-deep-dive>` for more details.
+
tvm/tir
-------
TIR contains the definition of the low-level program representations. We use `tir::PrimFunc` to represent functions that can be transformed by TIR passes.
-Besides the IR data structures, the tir module also defines a set of builtin intrinsics and their attributes via the common Op registry, as well as transformation passes in `tir/transform`.
+Besides the IR data structures, the tir module also includes:
+
+- A set of schedule primitives to control the generated code in ``tir/schedule``.
+- A set of builtin intrinsics in ``tir/tensor_intrin``.
+- A set of analysis passes to analyze the TIR functions in ``tir/analysis``.
+- A set of transformation passes to lower or optimize the TIR functions in ``tir/transform``.
+
+Please refer to the :ref:`TensorIR Deep Dive <tensor-ir-deep-dive>` for more details.
tvm/arith
---------
@@ -344,75 +323,28 @@ This module is closely tied to the TIR. One of the key problems in the low-level
arithmetic properties — the positiveness, variable bound, and the integer set that describes the iterator space. arith module provides
a collection of tools that do (primarily integer) analysis. A TIR pass can use these analyses to simplify and optimize the code.
-tvm/te
-------
-
-The name te stands for "tensor expression". This is a domain-specific language module that allows us to construct `tir::PrimFunc` variants quickly by writing tensor expressions.
-Importantly, a tensor expression itself is not a self-contained function that can be stored into IRModule. Instead, it is a fragment of IR that we can stitch together to build an IRModule.
+tvm/te and tvm/topi
+-------------------
-`te/schedule` provides a collection of scheduling primitives to control the function being generated. In the future, we might bring some of
-these scheduling components to the a `tir::PrimFunc` itself.
+TE stands for Tensor Expression. TE is a domain-specific language (DSL) for describing tensor computations. Importantly, a tensor expression
+itself is not a self-contained function that can be stored into IRModule. We can use ``te.create_prim_func`` to convert a tensor expression to a ``tir::PrimFunc``
+and then integrate it into the IRModule.
-.. toctree::
- :maxdepth: 1
-
- inferbound
- hybrid_script
-
-tvm/topi
---------
While possible to construct operators directly via TIR or tensor expressions (TE) for each use case it is tedious to do so.
-`topi` (Tensor operator inventory) provides a set of pre-defined operators (in TE or TIR) defined by
-numpy and found in common deep learning workloads. We also provide a collection of common schedule templates to obtain performant implementations across different target platforms.
-
-
-tvm/relay
----------
-Relay is the high-level functional IR used to represent full models. Various optimizations are defined in `relay.transform`. The Relay compiler defines multiple dialects,
-and each dialect is designed to support specific styles of optimization. Notable ones include QNN(for importing pre-quantized models), VM(for lowering to dynamic virtual machine),
-memory(for memory optimization).
-
-.. toctree::
- :maxdepth: 1
-
- relay_intro
- relay_op_strategy
- convert_layout
-
-
-tvm/autotvm
------------
+`topi` (Tensor operator inventory) provides a set of pre-defined operators defined by numpy and found in common deep learning workloads.
-AutoTVM and AutoScheduler are both components which automate search based program optimization. This is rapidly evolving and primarily consists of:
+tvm/meta_schedule
+-----------------
-- Cost models and feature extraction.
-- A record format for storing program benchmark results for cost model construction.
-- A set of search policies over program transformations.
+MetaSchedule is a system for automated search-based program optimization. It is designed to be a drop-in replacement for AutoTVM and AutoScheduler,
+and can be used to optimize TensorIR schedules. Note that MetaSchedule only works with static-shape workloads.
-Automated program optimization is still an active research field. As a result, we have attempted to modularize the design so that researchers may quickly modify a
-component or apply their own algorithms via the Python bindings, and
-customize the search and plugin their algorithms from the Python binding.
-
-.. toctree::
- :maxdepth: 1
-
- benchmark
-
-Frontends
----------
-Frontends ingest models from different frameworks into the TVM stack.
-:py:mod:`tvm.relay.frontend` is the namespace for model ingestion APIs.
-
-.. toctree::
- :maxdepth: 1
-
- frontend/tensorflow
+tvm/dlight
+----------
-microTVM
---------
-.. toctree::
- :maxdepth: 1
+DLight is a set of pre-defined, easy-to-use, and performant TIR schedules. DLight aims:
- microtvm_design
- microtvm_project_api
- model_library_format
+- Fully support **dynamic shape workloads**.
+- **Light weight**. DLight schedules provides tuning-free or (very few-shots tuning) schedule with reasonable performance.
+- **Robust**. DLight schedules are designed to be robust and general-purpose for a single rule. And if the rule is not applicable,
+ DLight not raise any error and switch to the next rule automatically.
diff --git a/docs/_sources/arch/inferbound.rst.txt b/docs/_sources/arch/inferbound.rst.txt
deleted file mode 100644
index cc516359bd..0000000000
--- a/docs/_sources/arch/inferbound.rst.txt
+++ /dev/null
@@ -1,763 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-.. _dev-InferBound-Pass:
-
-*******************************************
-InferBound Pass
-*******************************************
-
-
-The InferBound pass is run after normalize, and before ScheduleOps `build_module.py <https://github.com/apache/tvm/blob/main/python/tvm/driver/build_module.py>`_. The main job of InferBound is to create the bounds map, which specifies a Range for each IterVar in the program. These bounds are then passed to ScheduleOps, where they are used to set the extents of For loops, see `MakeLoopNest <https://github.com/apache/tvm/blob/main/src/te/operation/op_util.cc>`_, and to set the sizes of all [...]
-
-The output of InferBound is a map from IterVar to Range:
-
-.. code:: cpp
-
- Map<IterVar, Range> InferBound(const Schedule& sch);
-
-Therefore, let's review the Range and IterVar classes:
-
-.. code:: cpp
-
- namespace HalideIR {
- namespace IR {
- class RangeNode : public Node {
- public:
- Expr min;
- Expr extent;
- // remainder omitted
- };
- }}
-
- namespace tvm {
- class IterVarNode : public Node {
- public:
- Range dom;
- Var var;
- // remainder omitted
- };
- }
-
-Note that IterVarNode also contains a Range ``dom``. This ``dom`` may or may not have a meaningful value, depending on when the IterVar was created. For example, when ``tvm.compute`` is called, an `IterVar is created <https://github.com/apache/tvm/blob/main/src/te/operation/compute_op.cc>`_ for each axis and reduce axis, with dom's equal to the shape supplied in the call to ``tvm.compute``.
-
-On the other hand, when ``tvm.split`` is called, `IterVars are created <https://github.com/apache/tvm/blob/main/src/te/schedule/schedule_lang.cc>`_ for the inner and outer axes, but these IterVars are not given a meaningful ``dom`` value.
-
-In any case, the ``dom`` member of an IterVar is never modified during InferBound. However, keep in mind that the ``dom`` member of an IterVar is sometimes used as default value for the Ranges InferBound computes.
-
-We next review some TVM codebase concepts that are required to understand the InferBound pass.
-
-Recall that InferBound takes one argument, a Schedule. This schedule object, and its members, contains all information about the program being compiled.
-
-A TVM schedule is composed of Stages. Each stage has exactly one Operation, e.g., a ComputeOp or a TensorComputeOp. Each operation has a list of root_iter_vars, which in the case of ComputeOp, are composed of the axis IterVars and the reduce axis IterVars. Each operation can also contain many other IterVars, but all of them are related by the operations's list of IterVarRelations. Each IterVarRelation represents either a split, fuse or rebase in the schedule. For example, in the case of [...]
-
-
-.. code:: cpp
-
- namespace tvm {
- class ScheduleNode : public Node {
- public:
- Array<Operation> outputs;
- Array<Stage> stages;
- Map<Operation, Stage> stage_map;
- // remainder omitted
- };
-
- class StageNode : public Node {
- public:
- Operation op;
- Operation origin_op;
- Array<IterVar> all_iter_vars;
- Array<IterVar> leaf_iter_vars;
- Array<IterVarRelation> relations;
- // remainder omitted
- };
-
- class OperationNode : public Node {
- public:
- virtual Array<IterVar> root_iter_vars();
- virtual Array<Tensor> InputTensors();
- // remainder omitted
- };
-
- class ComputeOpNode : public OperationNode {
- public:
- Array<IterVar> axis;
- Array<IterVar> reduce_axis;
- Array<Expr> body;
- Array<IterVar> root_iter_vars();
- // remainder omitted
- };
- }
-
-Tensors haven't been mentioned yet, but in the context of TVM, a Tensor represents output of an operation.
-
-.. code:: cpp
-
- class TensorNode : public Node {
- public:
- // The source operation, can be None
- // This Tensor is output by this op
- Operation op;
- // The output index from the source operation
- int value_index;
- };
-
-In the Operation class declaration above, we can see that each operation also has a list of InputTensors. Thus the stages of the schedule form a DAG, where each stage is a node in the graph. There is an edge in the graph from Stage A to Stage B, if the operation of Stage B has an input tensor whose source operation is the op of Stage A. Put simply, there is an edge from A to B, if B consumes a tensor produced by A. See the diagram below. This graph is created at the beginning of InferBou [...]
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/docs/inferbound/stage_graph.png
- :align: center
-
-InferBound makes one pass through the graph, visiting each stage exactly once. InferBound starts from the output stages (i.e., the solid blue nodes in the graph above), and moves upwards (in the opposite direction of the edges). This is achieved by performing a reverse topological sort on the nodes of the graph. Therefore, when InferBound visits a stage, each of its consumer stages has already been visited.
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/docs/inferbound/inferbound_traversal.png
- :align: center
-
-The InferBound pass is shown in the following pseudo-code:
-
-.. code:: cpp
-
- Map<IterVar, Range> InferBound(const Schedule& sch) {
- Array<Operation> outputs = sch->get_outputs();
- G = CreateGraph(outputs);
- stage_list = sch->reverse_topological_sort(G);
- Map<IterVar, Range> rmap;
- for (Stage s in stage_list) {
- InferRootBound(s, &rmap);
- PassDownDomain(s, &rmap);
- }
- return rmap;
- }
-
-The InferBound pass has two interesting properties that are not immediately obvious:
-
-1. After InferBound visits a stage, the ranges of all IterVars in the stage will be set in ``rmap``.
-2. The Range of each IterVar is only set once in ``rmap``, and then never changed.
-
-So it remains to explain what InferBound does when it visits a stage. As can be seen in the pseudo-code above, InferBound calls two functions on each stage: InferRootBound, and PassDownDomain. The purpose of InferRootBound is to set the Range (in ``rmap``) of each root_iter_var of the stage. (Note: InferRootBound does not set the Range of any other IterVar, only those belonging to root_iter_vars). The purpose of PassDownDomain is to propagate this information to the rest of the stage's I [...]
-
-The remainder of the document dives into the details of InferRootBound and PassDownDomain. Since PassDownDomain is simpler to describe, we will cover it first.
-
-.. _IterVarHyperGraph:
-
-IterVar Hyper-graph
--------------------
-
-The InferBound pass traverses the stage graph, as described above. However, within each stage is another graph, whose nodes are IterVars. InferRootBound and PassDownDomain perform message-passing on these IterVar graphs.
-
-Recall that all IterVars of the stage are related by IterVarRelations. The IterVarRelations of a stage form a directed acyclic hyper-graph, where each node of the graph corresponds to an IterVar, and each hyper-edge corresponds to an IterVarRelation. We can also represent this hyper-graph as a DAG, which is simpler to visualize as shown below.
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/docs/inferbound/relations.png
- :align: center
-
-
-The above diagram shows the IterVar hyper-graph for one stage. The stage has one root_iter_var, ``i``. It has been split, and the resulting inner axis ``i.inner``, has been split again. The leaf_iter_vars of the stage are shown in green: ``i.outer``, ``i.inner.outer``, and ``i.inner.inner``.
-
-Message passing functions are named "PassUp" or "PassDown", depending on whether messages are passed from children to their parent in the DAG ("PassUp"), or from the parent to its children ("PassDown"). For example, the large arrow on the left-hand side of the diagram above, shows that PassDownDomain sends messages from the root IterVar ``i`` to its children ``i.outer`` and ``i.inner``.
-
-.. _PassDownDomain:
-
-PassDownDomain
---------------
-The purpose of PassDownDomain is to take the Ranges produced by InferRootBound for the root_iter_vars, and set the Ranges of all other IterVars in the stage.
-
-PassDownDomain iterates through the stage's IterVarRelations. There are three possible types of IterVarRelation: split, fuse, and rebase. The most interesting case (since it offers opportunity for improvement), is IterVarRelations representing splits.
-
-The Ranges of the inner and outer IterVars of the split are set based on the parent IterVar's known Range, as follows:
-
-.. code:: cpp
-
- rmap[split->inner] = Range::FromMinExtent(0, split->factor)
- rmap[split->outer] = Range::FromMinExtent(0, DivCeil(rmap[split->parent]->extent, split->factor))
-
-There is an opportunity here to tighten the bounds produced by InferBound, when ``split->factor`` does not evenly divide the parent's extent. Suppose the parent's extent is 20, and the split factor is 16. Then on the second iteration of the outer loop, the inner loop only needs to perform 4 iterations, not 16. If PassDownDomain could set the extent of ``split->inner`` to ``min(split->factor, rmap[split->parent]->extent - (split->outer * split->factor))``, then the extent of the inner var [...]
-
-For Fuse relations, the Range of the fused IterVar is set based on the known Ranges of the inner and outer IterVars, as follows:
-
-.. code:: cpp
-
- rmap[fuse->fused] = Range::FromMinExtent(0, rmap[fuse->outer]->extent * rmap[fuse->inner]->extent)
-
-
-InferRootBound
---------------
-
-Recall that InferBound calls InferRootBound, followed by :ref:`PassDownDomain` on each stage in the stage graph. The purpose of InferRootBound is to set the Range of each root_iter_var of the Stage's operation. These Ranges will be propagated to the rest of the stage's IterVars using :ref:`PassDownDomain`. Note that InferRootBound does not set the Range of any other IterVar, only those belonging to the stage's root_iter_vars.
-
-If the stage is an output stage or placeholder, InferRootBound simply sets the root_iter_var Ranges to their default values. The default Range for a root_iter_var is taken from the ``dom`` member of the IterVar (see the IterVarNode class declaration above).
-
-Otherwise, InferRootBound iterates through the consumers of the stage. IntSets are created for each of the consumer's IterVars, as follows. Phase 1) IntSets are initialized for the consumer's leaf_iter_vars, and propagated to the consumer's root_iter_vars by PassUpDomain (Phase 2). These IntSets are used to create TensorDom of the input tensors of the consumer stage (Phase 3). Finally, once all of the consumers have been processed, InferRootBound calls GatherBound, to set the Ranges of t [...]
-
-This process can seem complicated. One reason is that a stage can have more than one consumer. Each consumer has different requirements, and these must somehow be consolidated. Similarly, the stage may output more than one tensor, and each consumer only uses a particular subset of these tensors. Furthermore, even if a consumer uses a particular tensor, it may not use all elements of the tensor.
-
-As mentioned above, a consumer may only require a small number of elements from each tensor. The consumers can be thought of as making requests to the stage, for certain regions of its output tensors. The job of Phases 1-3 is to establish the regions of each output tensor that are required by each consumer.
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/docs/inferbound/inferbound_phases.png
- :align: center
-
-IntSets
-~~~~~~~
-
-During InferRootBound, Ranges are converted to IntSets, and message passing is performed over IntSets. Therefore, it is important to understand the difference between Ranges and IntSets. The name "IntSet" suggests it can represent an arbitrary set of integers, e.g., A = \{-10, 0, 10, 12, 13\}. This would certainly be more expressive than a Range, which only represents a set of contiguous integers, e.g., B = \{10,11,12\}.
-
-However, currently IntSets come in only three varieties: IntervalSets, StrideSets, and ModularSets. IntervalSets, similarly to Ranges, only represent sets of contiguous integers. A StrideSet is defined by a base IntervalSet, a list of strides, and a list of extents. However, StrideSet is unused, and ModularSet is only used by the frontend.
-
-Therefore, not all sets of integers can be represented by an IntSet in TVM currently. For example, set A in the example above can not be represented by an IntSet. However, in future the functionality of IntSet can be extended to handle more general kinds of integer sets, without requiring modification to users of IntSet.
-
-*InferBound is more complicated for schedules that contain compute_at. Therefore, we first explain InferBound for schedules that do not contain compute_at.*
-
-.. _Phase1:
-
-Phase 1: Initialize IntSets for consumer's leaf_iter_vars
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. code:: cpp
-
- /*
- * Input: Map<IterVar, Range> rmap: contains the Range for each IterVar of the consumer stage
- * Output: Map<IterVar, IntSet> up_state: contains an IntSet for each leaf_iter_var of the consumer
- */
-
-In Phase 1, IntSets for each of the consumer's leaf_iter_vars are created, based on the Ranges of the leaf_iter_vars from ``rmap``. Recall that the consumer has already been visited by InferBound, so all of its IterVars have known Ranges in ``rmap``.
-
-There are three cases:
-
-- Case 1: Extent of leaf var's Range is 1. In this case, the up_state for the leaf is just a single point, equal to the Range's min.
-- Case 2: *No relaxation is needed. In this case, the up_state for the leaf is just a single point, defined by the leaf var itself.*
-- Case 3: Relaxation is needed. In this case, the leaf's Range is simply converted to an IntSet.
-
-For simplicity, we assume the schedule does not contain thread axes. In this case, Case 2 is only relevant if the schedule contains compute_at. Please refer to the section :ref:`InferBoundCA`, for further explanation.
-
-.. _Phase2:
-
-Phase 2: Propagate IntSets from consumer's leaves to consumer's roots
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. code:: cpp
-
- /*
- * Input: Map<IterVar, IntSet> up_state: consumer leaf -> IntSet
- * Output: Map<IterVar, IntSet> dom_map: consumer root -> IntSet
- */
-
-The purpose of Phase 2 is to propagate the IntSet information from the consumer's leaf_iter_vars to the consumer's root_iter_vars. The result of Phase 2 is another map, ``dom_map``, that contains an IntSet for each of the consumer's root_iter_vars.
-
-Phase 2 begins by calling PassUpDomain, which visits the IterVarRelations of the consumer stage. In the case of a Split relation, PassUpDomain sets the up_state of the parent IterVar, based on the inner and outer IntSets, as follows:
-
-- Case 1: The Ranges of outer and inner IterVars match their ``up_state`` domains. In this case, set the parent's ``up_state`` by simply converting the parent's Range to an IntSet.
-- Case 2: *Otherwise, the parent's* ``up_state`` *is defined by evaluating* ``outer*f + inner + rmap[parent]->min``, *with respect to the* ``up_state`` *of outer and inner. Here, instead of using the Split relation's factor, TVM uses* ``f = rmap[inner]->extent``.
-
-Case 2 is only needed if the schedule contains compute_at. Please refer to the section :ref:`InferBoundCA` below, for further explanation.
-
-After PassUpDomain has finished propagating up_state to all IterVars of the consumer, a fresh map, from root_iter_vars to IntSet, is created. If the schedule does not contain compute_at, the IntSet for root_iter_var ``iv`` is created by the following code:
-
-.. code:: cpp
-
- dom_map[iv->var.get()] = IntSet::range(up_state.at(iv).cover_range(iv->dom));
-
-Note that if the schedule does not contain compute_at, Phases 1-2 are actually unnecessary. dom_map can be built directly from the known Ranges in rmap. Ranges simply need to be converted to IntSets, which involves no loss of information.
-
-.. _Phase3:
-
-Phase 3: Propagate IntSets to consumer's input tensors
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. code:: cpp
-
- /*
- * Input: Map<IterVar, IntSet> dom_map: consumer root -> IntSet
- * Output: Map<Tensor, TensorDom> tmap: output tensor -> vector<vector<IntSet>>
- */
-
-Note that the consumer's input tensors are output tensors of the stage InferBound is working on. So by establishing information about the consumer's input tensors, we actually obtain information about the stage's output tensors too: the consumers require certain regions of these tensors to be computed. This information can then be propagated through the rest of the stage, eventually obtaining Ranges for the stage's root_iter_vars by the end of Phase 4.
-
-The output of Phase 3 is tmap, which is a map containing all of the stage's output tensors. Recall that a Tensor is multi-dimensional, with a number of different axes. For each output tensor, and each of that tensor's axes, tmap contains a list of IntSets. Each IntSet in the list is a request from a different consumer.
-
-Phase 3 is accomplished by calling PropBoundToInputs on the consumer. PropBoundToInputs adds IntSets to tmap's lists, for all input Tensors of the consumer.
-
-The exact behavior of PropBoundToInputs depends on the type of the consumer's operation: ComputeOp, TensorComputeOp, PlaceholderOp, ExternOp, etc. Consider the case of TensorComputeOp. A TensorComputeOp already has a Region for each of its Tensor inputs, defining the slice of the tensor that the operation depends on. For each input tensor i, and dimension j, a request is added to tmap, based on the corresponding dimension in the Region:
-
-.. code:: cpp
-
- for (size_t j = 0; j < t.ndim(); ++j) {
- // i selects the Tensor t
- tmap[i][j].push_back(EvalSet(region[j], dom_map));
- }
-
-.. _Phase4:
-
-Phase 4: Consolidate across all consumers
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. code:: cpp
-
- /*
- * Input: Map<Tensor, TensorDom> tmap: output tensor -> vector<vector<IntSet>>
- * Output: Map<IterVar, Range> rmap: rmap is populated for all of the stage's root_iter_vars
- */
-
-Phase 4 is performed by GatherBound, whose behavior depends on the type of operation of the stage. We discuss the ComputeOp case only, but TensorComputeOp is the same.
-
-A ComputeOp has only a single output Tensor, whose axes correspond to the axis variables of the ComputeOp. The root_iter_vars of a ComputeOp include these axis variables, as well as the reduce_axis variables. If the root IterVar is an axis var, it corresponds to one of the axes of the output Tensor. GatherBound sets the Range of such a root IterVar to the union of all IntSets (i.e., union of all consumer requests) for the corresponding axis of the tensor. If the root IterVar is a reduce_ [...]
-
-.. code:: cpp
-
- // 'output' selects the output tensor
- // i is the dimension
- rmap[axis[i]] = arith::Union(tmap[output][i]).cover_range(axis[i]->dom);
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/docs/inferbound/gatherbound.png
- :align: center
-
-
-The union of IntSets is computed by converting each IntSet to an Interval, and then taking the minimum of all minimums, and the maximum of all of these interval's maximums.
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/docs/inferbound/union.png
- :align: center
-
-
-This clearly results in some unnecessary computation, i.e., tensor elements will be computed that are never used.
-
-Unfortunately, even if we're lucky and the IntervalSet unions do not produce unnecessary computation, the fact that GatherBound considers each dimension of the tensor separately can also cause unnecessary computation. For example, in the diagram below the two consumers A and B require disjoint regions of the 2D tensor: consumer A requires T[0:2, 0:2], and consumer B requires T[2:4, 2:4]. GatherBound operates on each dimension of the tensor separately. For the first dimension of the tenso [...]
-
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/docs/inferbound/gatherbound_problem.png
- :align: center
-
-.. _InferBoundCA:
-
-InferBound with compute_at
---------------------------
-
-If the schedule contains compute_at, Phases 1-2 of InferRootBound become more complex.
-
-Motivation
-~~~~~~~~~~
-
-**Ex. 1**
-
-Consider the following snippet of a TVM program:
-
-::
-
- C = tvm.compute((5, 16), lambda i, j : tvm.const(5, "int32"), name='C')
- D = tvm.compute((5, 16), lambda i, j : C[i, j]*2, name='D')
-
-This produces the following (simplified IR):
-
-::
-
- for i 0, 5
- for j 0, 16
- C[i, j] = 5
- for i 0, 5
- for j 0, 16
- D[i, j] = C[i, j]*2
-
-It's easy to see that stage D requires all (5,16) elements of C to be computed.
-
-**Ex. 2**
-
-However, suppose C is computed at axis j of D:
-
-::
-
- s = tvm.create_schedule(D.op)
- s[C].compute_at(s[D], D.op.axis[1])
-
-Then only a single element of C is needed at a time:
-
-::
-
- for i 0, 5
- for j 0, 16
- C[0] = 5
- D[i, j] = C[0]*2
-
-**Ex. 3**
-
-Similarly, if C is computed at axis i of D, only a vector of 16 elements of C are needed at a time:
-
-::
-
- for i 0, 5
- for j 0, 16
- C[j] = 5
- for j 0, 16
- D[i, j] = C[j]*2
-
-Based on the above examples, it is clear that InferBound should give different answers for stage C depending on where in its consumer D it is "attached".
-
-.. _AttachPaths:
-
-Attach Paths
-~~~~~~~~~~~~
-
-If stage C is computed at axis j of stage D, we say that C is *attached* to axis j of stage D. This is reflected in the Stage object by setting the following three member variables:
-
-.. code:: cpp
-
- class StageNode : public Node {
- public:
- // omitted
-
- // For compute_at, attach_type = kScope
- AttachType attach_type;
-
- // For compute_at, this is the axis
- // passed to compute_at, e.g., D.op.axis[1]
- IterVar attach_ivar;
-
- // The stage passed to compute_at, e.g., D
- Stage attach_stage;
-
- // omitted
- };
-
-Consider the above examples again. In order for InferBound to determine how many elements of C must be computed, it is important to know whether the computation of C occurs within the scope of a leaf variable of D, or above that scope. For example, in Ex. 1, the computation of C occurs *above* the scopes of all of D's leaf variables. In Ex. 2, the computation of C occurs *within* the scope of all of D's leaf variables. In Ex. 3, C occurs within the scope of D's i, but above the scope [...]
-
-CreateAttachPath is responsible for figuring out which scopes contain a stage C. These scopes are ordered from innermost scope to outermost. Thus for each stage CreateAttachPath produces an "attach path", which lists the scopes containing the stage, from innermost to outermost scope. In Ex. 1, the attach path of C is empty. In Ex. 2, the attach path of C contains {j, i}. In Ex. 3, the attach path of C is {i}.
-
-The following example clarifies the concept of an attach path, for a more complicated case.
-
-**Ex. 4**
-
-::
-
- C = tvm.compute((5, 16), lambda i, j : tvm.const(5, "int32"), name='C')
- D = tvm.compute((4, 5, 16), lambda di, dj, dk : C[dj, dk]*2, name='D')
- s = tvm.create_schedule(D.op)
- s[C].compute_at(s[D], D.op.axis[2])
-
-Here is the IR after ScheduleOps (note that loops with extent 1 have been preserved, using the ``debug_keep_trivial_loop`` argument of ScheduleOps):
-
-::
-
- realize D([0, 4], [0, 5], [0, 16]) {
- produce D {
- for (di, 0, 4) {
- for (dj, 0, 5) {
- for (dk, 0, 16) {
- realize C([dj, 1], [dk, 1]) {
- produce C {
- for (i, 0, 1) {
- for (j, 0, 1) {
- C((i + dj), (j + dk)) =5
- }
- }
- }
- D(di, dj, dk) =(C(dj, dk)*2)
- }
- }
- }
- }
- }
- }
-
-In this case, the attach path of C is {dk, dj, di}. Note that C does not use di, but di still appears in C's attach path.
-
-**Ex. 5**
-
-Compute_at is commonly applied after splitting, but this can be handled very naturally given the above definitions. In the example below, the attachment point of C is j_inner of D. The attach path of C is {j_inner, j_outer, i}.
-
-::
-
- C = tvm.compute((5, 16), lambda i, j : tvm.const(5, "int32"), name='C')
- D = tvm.compute((5, 16), lambda i, j : C[i, j]*2, name='D')
- s = tvm.create_schedule(D.op)
- d_o, d_i = s[D].split(D.op.axis[1], factor=8)
- s[C].compute_at(s[D], d_i)
-
-The IR in this case looks like:
-
-::
-
- for i 0, 5
- for j_outer 0, 2
- for j_inner 0, 8
- C[0] = 5
- D[i, j_outer*8 + j_inner] = C[0]*2
-
-Building an Attach Path
-~~~~~~~~~~~~~~~~~~~~~~~
-
-We continue to refer to stages C and D, as introduced in the previous section. The CreateAttachPath algorithm builds the attach path of a stage C as follows. If C does not have attach_type ``kScope``, then C has no attachment, and C's attach path is empty. Otherwise, C is attached at attach_stage=D. We iterate through D's leaf variables in top-down order. All leaf variables starting from C.attach_ivar and lower are added to C's attach path. Then, if D is also attached somewhere, e.g., to [...]
-
-In the example below, C is attached at D, and D is attached at E.
-
-::
-
- C = tvm.compute((5, 16), lambda ci, cj : tvm.const(5, "int32"), name='C')
- D = tvm.compute((5, 16), lambda di, dj : C[di, dj]*2, name='D')
- E = tvm.compute((5, 16), lambda ei, ej : D[ei, ej]*4, name='E')
- s = tvm.create_schedule(E.op)
- s[C].compute_at(s[D], D.op.axis[1])
- s[D].compute_at(s[E], E.op.axis[1])
-
-With ``debug_keep_trivial_loop=True``, the attach path of C is {dj, di, ej, ei}, and the attach path of D is {ej, ei}:
-
-::
-
- // attr [D] storage_scope = "global"
- allocate D[int32 * 1]
- // attr [C] storage_scope = "global"
- allocate C[int32 * 1]
- produce E {
- for (ei, 0, 5) {
- for (ej, 0, 16) {
- produce D {
- for (di, 0, 1) {
- for (dj, 0, 1) {
- produce C {
- for (ci, 0, 1) {
- for (cj, 0, 1) {
- C[(ci + cj)] = 5
- }
- }
- }
- D[(di + dj)] = (C[(di + dj)]*2)
- }
- }
- }
- E[((ei*16) + ej)] = (D[0]*4)
- }
- }
- }
-
-InferBound with compute_at
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Now that the concept of an attach path has been introduced, we return to how InferBound differs if the schedule contains compute_at. The only difference is in InferRootBound, :ref:`Phase1` and :ref:`Phase2`.
-
-In InferRootBound, the goal is to determine Ranges for the root_iter_vars of a particular stage, C. Phases 1-2 of InferRootBound assign IntSets to the leaf IterVars of C's consumers, and then propagate those IntSets up to the consumers' root_iter_vars.
-
-If there are no attachments, the Ranges already computed for the consumer's variables define how much of C is needed by the consumer. However, if the stage is actually inside the scope of one of the consumer's variables j, then only a single point within the Range of j is needed at a time.
-
-.. _Phase1CA:
-
-Phase 1: Initialize IntSets for consumer's leaf_iter_vars
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. code:: cpp
-
- /*
- * Input: Map<IterVar, Range> rmap: contains the Range for each IterVar of the consumer stage
- * Output: Map<IterVar, IntSet> up_state: contains an IntSet for each leaf_iter_var of the consumer
- */
-
-In Phase 1, IntSets for each of the consumer's leaf_iter_vars are created, based on the Ranges of the leaf_iter_vars from rmap. Recall that the consumer has already been visited by InferBound, so all of its IterVars have known Ranges in rmap.
-
-There are three cases:
-
-- Case 1: Extent of leaf var's Range is 1. In this case, the up_state for the leaf is just a single point, equal to the Range's min.
-- Case 2: No relaxation is needed. In this case, the up_state for the leaf is just a single point, defined by the leaf var itself.
-- Case 3: Relaxation is needed. In this case, the leaf's Range is simply converted to an IntSet.
-
-Case 2 occurs if we encounter the attachment point of stage C in the consumer. For this attach_ivar, and all higher leaf variables of the consumer, Case 2 will be applied. This ensures that only a single point within the Range of the leaf variable will be requested, if C is inside the leaf variable's scope.
-
-.. _Phase2CA:
-
-Phase 2: Propagate IntSets from consumer's leaves to consumer's roots
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. code:: cpp
-
- /*
- * Input: Map<IterVar, IntSet> up_state: consumer leaf -> IntSet
- * Output: Map<IterVar, IntSet> dom_map: consumer root -> IntSet
- */
-
-Phase 2 begins by calling PassUpDomain, which visits the IterVarRelations of the consumer stage. In the case of a Split relation, PassUpDomain sets the up_state of the parent IterVar, based on the inner and outer IntSets, as follows:
-
-- Case 1: The Ranges of outer and inner IterVars match their ``up_state`` domains. In this case, set the parent's ``up_state`` by simply converting the parent's Range to an IntSet.
-- Case 2: Otherwise, the parent's ``up_state`` is defined by evaluating ``outer*f + inner + rmap[parent]->min``, with respect to the ``up_state`` of outer and inner. Here, instead of using the Split relation's factor, TVM uses* ``f = rmap[inner]->extent``.
-
-
-Now, because the schedule contains compute_at, it is possible for Case 2 to apply. This is because the leaf IntSets may now be initialized to a single point within their Range (Case 2 of :ref:`Phase1CA`), so the IntSets will no longer always match the Ranges.
-
-After PassUpDomain has finished propagating up_state to all IterVars of the consumer, a fresh map, from root_iter_vars to IntSet, is created. If the stage is not attached to the current consumer, then for each variable iv in the consumer's attach_path, iv's Range is added to a ``relax_set``. The root variables of the stage are evaluated with respect to this ``relax_set``.
-
-This is to handle cases like the following example, where C is not attached anywhere, but its consumer D is attached in stage E. In this case, D's attach_path, {ej, ei} must be considered when determining how much of C must be computed.
-
-::
-
- C = tvm.compute((5, 16), lambda ci, cj : tvm.const(5, "int32"), name='C')
- D = tvm.compute((5, 16), lambda di, dj : C[di, dj]*2, name='D')
- E = tvm.compute((5, 16), lambda ei, ej : D[ei, ej]*4, name='E')
- s = tvm.create_schedule(E.op)
- s[D].compute_at(s[E], E.op.axis[1])
-
-
-::
-
- for ci 0, 5
- for cj 0, 16
- C[ci, cj] = 5
- for ei 0, 5
- for ej 0, 16
- D[0] = C[ei, ej]*2
- E[ei, ej] = D[0]*4
-
-Limitations of PassUpDomain
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-This section describes known limitations of PassUpDomain. These limitations affect the Ranges produced by InferBound, as well as other users of PassUpDomain such as ``tensorize``.
-
-**Ex. 6**
-
-Above, we discussed the behavior of PassUpDomain on Split relations only. In the following example, the schedule contains ``fuse`` in addition to ``split``. In the TVM program below, the operation C has two axes that are fused, and then the fused axis is split. Note that all tensors are originally of shape ``(4, 4)`` and the fused axis is split by factor ``4`` as well. Therefore, it would be natural to assume that the effect of the fuse is simply undone by the split. However, this is not [...]
-
-::
-
- import tvm
- from tvm import te
-
- n = 4
- m = 4
-
- A = te.placeholder((n, m), name='A')
- B = te.compute((n, m), lambda bi, bj: A[bi, bj]+2, name='B')
- C = te.compute((n, m), lambda ci, cj: B[ci, cj]*3, name='C')
-
- s = te.create_schedule(C.op)
-
- fused_axes = s[C].fuse(C.op.axis[0], C.op.axis[1])
- xo, xi = s[C].split(fused_axes, 4)
-
- s[B].compute_at(s[C], xo)
-
- print(tvm.lower(s, [A, C], simple_mode=True))
-
-The output of this program is shown below. Notice that all 16 elements of B are computed every time through the outer loop, even though C only uses 4 of them.
-
-::
-
- // attr [B] storage_scope = "global"
- allocate B[float32 * 16]
- produce C {
- for (ci.cj.fused.outer, 0, 4) {
- produce B {
- for (bi, 0, 4) {
- for (bj, 0, 4) {
- B[((bi*4) + bj)] = (A[((bi*4) + bj)] + 2.000000f)
- }
- }
- }
- for (ci.cj.fused.inner, 0, 4) {
- C[((ci.cj.fused.outer*4) + ci.cj.fused.inner)] = (B[((ci.cj.fused.outer*4) + ci.cj.fused.inner)]*3.000000f)
- }
- }
- }
-
-This is in contrast to the following IR, which is produced by modifying the above program by deleting the fuse and split, and replacing the compute_at with ``s[B].compute_at(s[C], C.op.axis[0])``. Note that in the IR below, only 4 elements of B are computed at a time, as desired. The size of buffer B is also smaller.
-
-::
-
- // attr [B] storage_scope = "global"
- allocate B[float32 * 4]
- produce C {
- for (ci, 0, 4) {
- produce B {
- for (bj, 0, 4) {
- B[bj] = (A[((ci*4) + bj)] + 2.000000f)
- }
- }
- for (cj, 0, 4) {
- C[((ci*4) + cj)] = (B[cj]*3.000000f)
- }
- }
- }
-
-This example demonstrates that contrary to what we expect, the split does not simply undo the fuse. So what causes the difference? Why is the entire tensor B re-computed 4 times, when only a single row is actually needed at a time?
-
-Determining the amount of B that must be computed is the responsibility of InferBound. However, the Ranges returned by InferBound for B's root_iter_vars are too large in this case: ``[0, 4]`` for both ``bi`` and ``bj``. This occurs because of a limitation in PassUpDomain on Fuse relations, which we explain next.
-
-When InferRootBound is working on stage B, it visits B's consumer stage C to find out how much of B is requested by C. C has root_iter_vars ci and cj, which have been fused and then split. This results in the following :ref:`IterVarHyperGraph` for stage C.
-
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/docs/inferbound/passupdomain_problem.png
- :align: center
-
-
-
-We trace the execution of InferRootBound on stage B. Recall that :ref:`Phase1CA` of InferRootBound involves setting the IntSets for all leaf_iter_vars of B's consumer stage C. In this case, C's leaf_iter_vars are ``ci.cj.fused.outer`` and ``ci.cj.fused.inner``. Since B is attached at ``ci.cj.fused.outer``, ``ci.cj.fused.inner`` must be relaxed but ``ci.cj.fused.outer`` is a single point. The IntSets of C's leaf_iter_vars, after :ref:`Phase1CA`, are shown in the following table.
-
-+----------------------+---------------------------------------------------+
-| IterVar | IntSet after Phase 1 |
-+======================+===================================================+
-| ``ci.cj.fused.inner``|``[0, (min(4, (16 - (ci.cj.fused.outer*4))) - 1)]``|
-+----------------------+---------------------------------------------------+
-| ``ci.cj.fused.outer``| ``[ci.cj.fused.outer, ci.cj.fused.outer]`` |
-+----------------------+---------------------------------------------------+
-
-In :ref:`Phase2CA` of InferRootBound, PassUpDomain is called on all of C's IterVarRelations in bottom-up order.
-
-PassUpDomain is called on C's Split node first. Case 2 of PassUpDomain applies, because the IntSet of ``ci.cj.fused.outer`` is just a single point, and doesn't equal its Range (as previously computed by InferBound on stage C). PassUpDomain therefore sets the IntSet of ``ci.cj.fused`` based on the IntSets of ``ci.cj.fused.inner`` and ``ci.cj.fused.outer``, as shown in row 3 of the following table.
-
-+----------------------+--------------------------------------------------------------------------------------------------+
-| IterVar | IntSet after PassUpDomain on SplitNode |
-+======================+==================================================================================================+
-| ``ci.cj.fused.inner``| ``[0, (min(4, (16 - (ci.cj.fused.outer*4))) - 1)]`` |
-+----------------------+--------------------------------------------------------------------------------------------------+
-| ``ci.cj.fused.outer``| ``[ci.cj.fused.outer, ci.cj.fused.outer]`` |
-+----------------------+--------------------------------------------------------------------------------------------------+
-| ``ci.cj.fused`` | ``[(ci.cj.fused.outer*4), ((ci.cj.fused.outer*4) + (min(4, (16 - (ci.cj.fused.outer*4))) - 1))]``|
-+----------------------+--------------------------------------------------------------------------------------------------+
-
-After PassUpDomain is called on the Split node, it is called on the Fuse node.
-
-- Case 1: the Range of IterVar ``fused`` (i.e., as previously calculated by InferBound) is equal to its IntSet
-- Case 2: the IntSet of IterVar ``fused`` is a single point
-- Case 3: otherwise
-
-In our case, the Range of ``ci.cj.fused``, is [0, 16). This is not equal to the IntSet of ``ci.cj.fused``, which has extent at most 4 (see row 3 of the table above). Therefore Case 1 does not apply. Case 2 doesn't apply either, since the IntSet of ``ci.cj.fused`` is not a single point. Therefore, only the default Case 3 applies.
-
-Unfortunately in Case 3, PassUpDomain conservatively applies a "fallback inference rule", i.e., it just returns IntSets equal to the Ranges of ``ci`` and ``cj``. Since C is the output stage of the schedule, we know that InferBound will have set the Ranges of the root_iter_vars of C (i.e., ``ci`` and ``cj``) to their original dimensions (i.e., the ``dom`` value of their IterVars). The resulting output of PassUpDomain for ``ci`` and ``cj`` is shown in the last two rows of the table below.
-
-+----------------------+--------------------------------------------------------------------------------------------------+
-| IterVar | IntSet after PassUpDomain on FuseNode |
-+======================+==================================================================================================+
-| ``ci.cj.fused.inner``| ``[0, (min(4, (16 - (ci.cj.fused.outer*4))) - 1)]`` |
-+----------------------+--------------------------------------------------------------------------------------------------+
-| ``ci.cj.fused.outer``| ``[ci.cj.fused.outer, ci.cj.fused.outer]`` |
-+----------------------+--------------------------------------------------------------------------------------------------+
-| ``ci.cj.fused`` |``[(ci.cj.fused.outer*4), ((ci.cj.fused.outer*4) + (min(4, (16 - (ci.cj.fused.outer*4))) - 1))]`` |
-+----------------------+--------------------------------------------------------------------------------------------------+
-| ``ci`` | ``[0, 4]`` |
-+----------------------+--------------------------------------------------------------------------------------------------+
-| ``cj`` | ``[0, 4]`` |
-+----------------------+--------------------------------------------------------------------------------------------------+
-
-This is enough to guarantee that consumer C requests *all* elements of B: the IntSets of ``ci`` and ``cj`` become requests from consumer C to the output tensors of stage B (via PropBoundToInputs in :ref:`Phase3` and GatherBound in :ref:`Phase4`).
-
-This example shows that schedules containing a split of fused axes are difficult to handle in TVM. The source of the difficulty is similar to the limitations of GatherBound. The region of tensor B requested by a consumer C must be a single rectangular region of B. Or, if B has more than two dimensions, the region of B must be expressible as an independent Range for each of its axes.
-
-If the split factor is 4, or 8, in the above example, the region of B needed in each iteration of the outer loop is rectangular.
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/docs/inferbound/passupdomain_div.png
- :align: center
-
-However, if the split factor is changed from 4 to 3 in the example above, it is easy to see that the region of B that C needs can no longer be described by an independent Range for each of its axes.
-
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/docs/inferbound/passupdomain_nodiv.png
- :align: center
-
-The best that can be done with rectangular regions is shown in the following diagram. The orange regions are the minimum rectangular regions covering the region of B that needs to be computed, at each iteration of the outer loop.
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/docs/inferbound/passupdomain_min.png
- :align: center
diff --git a/docs/_sources/arch/microtvm_design.rst.txt b/docs/_sources/arch/microtvm_design.rst.txt
deleted file mode 100644
index f9c06c10b6..0000000000
--- a/docs/_sources/arch/microtvm_design.rst.txt
+++ /dev/null
@@ -1,357 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-.. http://www.apache.org/licenses/LICENSE-2.0
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-.. _microtvm-design:
-
-**************************
-microTVM Design Document
-**************************
-
-.. contents:: Table of Contents
- :depth: 3
-
-Background
-===========
-
-TVM is a model deployment framework that has demonstrated good performance across a wide range of
-models on traditional operating systems. Given TVM's layered approach to compilation, it is a
-natural extension to target bare metal devices. While most of the compilation flow does not need to
-change for a proof-of-concept implementation on such devices, the runtime cannot depend on:
-
-* **Virtual Memory**, and by extension any system-provided ``malloc``. Additionally, bare metal
- devices typically have very limited memory (measured in KB). Because of this, libraries designed
- for such platforms typically need to be more judicious in using memory, and need to release
- memory when it is not in use.
-* Traditional OS abstractions, such as **files**, **libraries**, and **kernel functions**. Some
- projects implement support for these, but they are by no means standard.
-* Support for programming languages other than **C**.
-
-Such changes require a different approach from the TVM C++ runtime typically used on traditional
-Operating Systems.
-
-Typical Use
-===========
-
-This section discusses our vision of the "typical" microTVM use case. Each component used to achieve
-this typical use case is intended to be designed for flexibility, but this unifying vision serves to
-motivate the inclusion of each part of the design.
-
-.. figure:: https://raw.githubusercontent.com/tvmai/web-data/main/images/dev/microtvm_workflow.svg
- :align: center
- :width: 85%
-
-The parts of this process are described below:
-
-#. **Model Import**. The user imports an existing model or describes a new model to TVM, producing a
- *Relay module*.
-
-#. **Model Transformations**. The user can apply transformations, such as quantization, to the
- model. After each transformation, the user should still have a Relay module.
-
-#. **Compilation** (Scheduling and Code Generation). TVM implements each operator into Tensor IR by
- assigning a schedule and schedule configuration to each Relay operator. Then, code (C source or
- compiled object) is generated for each operator.
-
-#. **Integration**. The generated code is integrated along with the TVM C Runtime library into a
- user-supplied binary project. In some cases (such as when the project is standardized across
- multiple SoC/development boards), this process is handled automatically.
-
-#. **Deployment**. The project is built and the residual firmware binary is flashed onto the device.
- Model inference is driven either by TVM using an on-device RPC server, or on the device using the
- on-device Graph Executor.
-
-Design Goals
-============
-
-microTVM aims to achieve these design goals:
-
-1. **Portable Code**. microTVM can translate any Relay model into C code that can compile with only
- a C standard library.
-2. **Minimal Overhead**. microTVM generates target-specific, highly optimized code. As much overhead
- from the runtime should be removed.
-3. **Accessible Code**. microTVM considers C source code as a first-class output mechanism so that
- it is easier for a firmware engineer to understand and tweak.
-
-Overview
-========
-
-microTVM requires changes at all levels of the TVM compiler stack. The following sub-sections enumerate
-these changes at a high level, and follow-on sections discuss the specifics in more detail.
-
-Modeling Target Platforms
--------------------------
-
-TVM's search-based optimization approach allows it to largely avoid system-level modeling of targets
-in favor of experimental results. However, some modeling is necessary in order to ensure TVM is
-comparing apples-to-apples search results, and to avoid wasting time during the search by attempting
-to compile invalid code for a target.
-
-microTVM models these parts of the target:
-
-* The CPU used, through the ``-mcpu`` and ``-march`` target flags.
-* The presence or absence of accelerators, through the device components of the target (Currently
- only the absence of accelerators can be expressed, but this mechanism should extend well).
-
-microTVM aims to model these parts of the target in the future:
-
-* Memory, modeled as a set of disjoint memory spaces, each with a label and size and prefetch/flush
- behavior. Some memory may be shared with accelerators.
-* Target runtime configuration (i.e. clock tree configuration, clock speed, etc). This is intended
- only to contribute to the AutoTVM schedule key and not for any other use.
-
-At this time, TVM does not intend to model:
-
-* Size, type, or relationship of caches, with the exception of prefetching or cache flushing.
-
-
-TVM Targets for microTVM
--------------------------
-
-A central data structure in the compilation process is the ``tvm::target::Target`` class. TVM uses
-Target to decide which TIR schedules to enable and how to configure the code generator. The Target
-class should also uniquely identify the generated code for a particular operator, as autotuning
-logs use it to rank measured performance (but see Future Work).
-
-Targets are currently represented as strings structured similarly to command-line arguments. An
-example target is shown below:
-
- ``c -keys=arm_cpu -mcpu=cortex-m7 -model=stm32f746xx``
-
-The relevant parts to microTVM are:
-
- * Code generator (``llvm`` or ``c``)
- * ``-mcpu=cortex-m7``: used by TOPI to enable Cortex-M schedules, and, when the C source code
- generator is selected, included in the output as a comment to help identify the code and
- configure the downstream C compiler.
-
-Runtime and Executor configuration for microTVM
------------------------------------------------
-
-When using microTVM, it's important to use the C Runtime (``Runtime('crt')``), which is the runtime that works best on micro devices rather than the more dynamic C++ Runtime. Alongside this, there are two executors which you could use in combination with the C runtime:
-
-* ``Executor("aot")`` - The Ahead of Time (AOT) executor precompiles the network into a runnable function which you can add directly into your micro application
-* ``Executor("graph", {"link-params": True})`` - The Graph executor provides a JSON representation of your network and requires the C Runtime's system library to be generated to find functions in the function registry (``Runtime("crt", {"system-lib": True})``). ``{"link-params":True}`` enables parameters to be linked into the generated files rather than provided externally.
-
-These are specified when building a runtime module: ``relay.build(..., runtime=..., executor=...)``.
-
-Writing Schedules for microTVM
-------------------------------
-
-For operations scheduled on the CPU, microTVM initially plans to make use of specialized
-instructions and extern (i.e. hand-optimized) functions to achieve good performance. In TVM, this
-approach is generally accomplished through tensorization, in which TVM breaks a computation into
-small pieces, and a TIR extern function accelerates each small piece.
-
-TVM currently accommodates both approaches using ``tir.call_extern``. First, a pragma is attached to
-the schedule defining the extern function in portable C.
-
- ``sched[output].pragma(n, "import_c", "void call_asm(int32_t* a, int32_t* b) { /* ... */ }")``
-
-Next, ``tensorize`` is used to split the computation.
-
- ``sched[output].tensorize(owi, gemm)``
-
-There are a couple of caveats to this approach, all which could be resolved by linking generated
-code against external libraries:
-
-* Inline assembly is compiler-specific. While Clang and GCC have standardized on one syntax, this
- may not be portable to other compilers. SDKs solve this by conditionally including a header file
- depending on the compiler being used. However, taking this approach means that the generated code
- needs additional compiler flags (i.e. ``-Isystempath/to/header``).
-* It may be helpful to reference helper functions from the generated code (e.g. to inline common
- sequences of hand-optimized assembly).
-* Finally, the extern function invoked may be wholly written in an external library. If those
- functions can be wholly inlined, this caveat is the same as the previous. If not, then additional
- C code needs to be compiled and linked against the operator.
-
-At present, microTVM presumes that all eligible schedules can be compiled. This means that the user-
-supplied project (see next section) must include all libraries that are used by the generated code.
-When not using autotuning, TVM randomly chooses a fallback schedule, so all libraries would need to
-be supported. When using autotuning, TVM selects the best-performing schedule, so only that library
-is needed. There isn't currently a way to force TVM to pick a particular schedule outside of
-autotuning logs, but that would be a good addition.
-
-Finally, when using the ``llvm`` backend, the process is similar except that LLVM bitcode is included
-in the generated code (with an ``import_llvm`` pragma). LLVM bitcode provides a portable way to call
-inline assembly. However, it may be more complex to call external C functions, and helper functions
-are of course not easy to use from LLVM bitcode.
-
-Executing Models
-----------------
-
-The TVM compiler traditionally outputs three pieces:
-
-1. Model operator implementations, as discussed above;
-2. A model execution graph, encoded as JSON; and
-3. Simplified parameters.
-
-To correctly execute the model, a Graph Executor needs to reconstruct the graph in memory, load the
-parameters, and then invoke the operator implementations in the correct order.
-
-microTVM supports two ways to do this:
-
-1. **Host-Driven**. The Graph Executor can run on the host and carry out execution by issuing
- commands to the device using an RPC link with a UART-like transport.
-2. **Standalone**. A C Graph Executor is available to be compiled on-device, but it is not
- particularly memory efficient. This way enables standalone execution without any attached host.
-
-Host-Driven is designed for experimenting with models on-device and, like AutoTVM, uses the RPC server to
-drive computation on-device. Standalone is intended for deployment.
-
-Host-Driven Execution
-^^^^^^^^^^^^^^^^^^^^^
-
-In Host-Driven execution, the firmware binary is the following:
-
-1. Generated operator implementations from TVM.
-2. The TVM C runtime.
-3. SoC-specific initialization.
-4. The TVM RPC server.
-5. (optional) Simplified Parameters.
-
-This firmware image is flashed onto the device and a GraphExecutor instance is created on the host.
-The GraphExecutor drives execution by sending RPC commands over a UART:
-
-.. figure:: https://raw.githubusercontent.com/tvmai/web-data/main/images/dev/microtvm_host_driven.svg
- :align: center
- :width: 85%
-
-Standalone Execution
-^^^^^^^^^^^^^^^^^^^^
-
-In Standalone execution, the GraphExecutor is instantiated on device:
-
-.. figure:: https://raw.githubusercontent.com/tvmai/web-data/main/images/dev/microtvm_standalone.svg
- :align: center
- :width: 85%
-
-microTVM Firmware
-------------------
-
-We can now discuss how microTVM firmware should behave. An important task common to both model
-execution strategies is configuring the SoC to match the way it performs in production. microTVM
-considers this task project- and SoC-dependent. Whether for AutoTVM, host-driven model inference, or
-in standalone deployment, the user is expected to supply a project whose main() does the following:
-
-1. Configure the SoC to match deployment performance.
-2. Initialize the TVM C Runtime.
-
-When configuring for host-driven inference or AutoTVM, the remaining tasks are well-defined:
-
-3. Initialize a transport (i.e. a UART) for use with the TVM RPC server.
-4. Launch the TVM RPC Server.
-
-When configuring for standalone deployment, the firmware needs to:
-
-1. Instantiate the system library by calling the ``runtime.SystemLib`` PackedFunc.
-2. Instantiate a GraphExecutor passing the system library module.
-3. Configure parameters and inputs as needed.
-4. Run the model.
-
-Parts of a microTVM Binary
---------------------------
-
-To summarize, a microTVM firwmare binary image must contain these parts:
-
-1. Operator implementations, produced by TVM.
-2. The TVM C runtime library, supplied by TVM as a static library.
-3. SoC Initialization, supplied by the user.
-
-For Host-driven model execution, firmware also needs:
-
-4. The TVM RPC Server library.
-
-For Standalone model execution, firmware also needs:
-
-4. The TVM C GraphExecutor library, supplied by TVM as a static library.
-5. The remaining compiler outputs (Simplified Parameters and Graph JSON).
-
-The Automated Build Flow
-------------------------
-
-Once code generation is complete, ``tvm.relay.build`` returns a ``tvm.runtime.Module`` and the
-user can save the generated C source or binary object to a ``.c`` or ``.o`` file. From this point, TVM
-can theoretically step back and the user can compile and run the code separately.
-
-However, for AutoTVM, TVM needs some automated flow to handle the following tasks:
-
-1. Integrate operator implementations, the TVM C Runtime library, and the TVM RPC Server library into the
- firmware project containing user-supplied SoC Initialization.
-2. Build the resulting project.
-3. Program the built firmware onto a (specific) attached device.
-4. Identify the serial port or other transport to be used by TVM to drive remote execution.
-
-At present, TVM expects the user to supply an implementation of the ``tvm.micro.Compiler``,
-``tvm.micro.Flasher``, and ``tvm.micro.Transport`` interfaces. TVM then:
-
-1. Builds each piece separately as a library.
-2. Builds the libraries into a binary firmware image.
-3. Programs the firmware image onto an attached device.
-4. Opens a serial port to serve as the RPC server transport.
-
-This design was chosen to reduce build times for microTVM (the common libraries need to be built
-only once per candidate operator implemmentation). In practice, these projects are extremely small
-and compile relatively quickly. Compared with the added complexity of this tighter build integration
-with TVM, the performance gains are likely not worth it. A future design will consolidate the build
-tasks into a single step and narrow the interface to provide a better integration.
-
-Measuring operator performance
-------------------------------
-
-The TVM C runtime depends on user-supplied functions to measure time on-device. Users should implement
-``TVMPlatformTimerStart`` and ``TVMPlatformTimerStop``. These functions should measure wall clock time, so there
-are some pitfalls in implementing these functions:
-
-1. If the CPU could halt or sleep during a computation (i.e. if it is being done on an accelerator),
- a cycle counter should likely not be used as these tend to stop counting while the CPU is asleep.
-2. The granularity of these functions can be relaxed as needed to extend the range of the timer
- device. However, if granularity is too coarse, a sub-optimal schedule may be used.
-3. An error should be raised if the timer overflows.
-4. The timer should not interrupt computation unless absolutely necessary. Doing so may affect the
- accuracy of the results.
-5. Calibrating the output against a wall clock is ideal, but it will likely be too cumbersome. A
- future PR could enable some characterization of the platform timer by, e.g., measuring the internal
- oscillator against a reference such as an external crystal.
-
-Future Work
-===========
-
-Ahead-of-Time Runtime
-----------------------
-
-A limitation of the Graph Executor is the amount of memory overhead required in parsing the JSON.
-The current implementation contributes significantly to the dynamic memory usage of microTVM,
-limiting its utility. An ahead-of-time runtime can avoid the need for any Graph JSON parsing and
-improve inference speed by generating C code to call the generated operator implementations directly
-rather than relying on a data-driven approach with the Graph Executor.
-
-Memory Planning
-----------------
-
-The current memory planner attempts to limit the number of ``TVMBackendDeviceAlloc()`` calls
-issued for intermediate tensors only. Because scratchpads can vary widely, and because the planner
-coalesces memory allocations within 16x of each other, this strategy typically results in high
-peak memory usage.
-
-Heterogeneous Execution
------------------------
-
-Newer Cortex-M SoCs can contain multiple CPUs and onboard ML accelerators.
-
-
-Autotuning Target
------------------
-
-As discussed previously,
diff --git a/docs/_sources/arch/microtvm_project_api.rst.txt b/docs/_sources/arch/microtvm_project_api.rst.txt
deleted file mode 100644
index 381b57876a..0000000000
--- a/docs/_sources/arch/microtvm_project_api.rst.txt
+++ /dev/null
@@ -1,150 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-.. _microtvm_project_api:
-
-microTVM Project API
-====================
-
-About microTVM Project API
---------------------------
-
-The microTVM Project API allows TVM to automatically run models on
-unconventional or embedded platforms. It allows platforms to define a standard
-function to integrate TVM compiler output with boilerplate platform-specific
-code, producing a runnable **Project**. Project API then further defines
-functions to build that project, program compatible devices accessible from the
-TVM machine, and communicate with the running code so that TVM can perform
-host-driven inference and autotuning.
-
-There are many cases where it might be desirable simply to invoke microTVM as a
-tool from your platform's build process. Indeed, for the average firmware
-developer, this is likely to be all they need. However, there are a couple of
-use cases when you may want to teach microTVM how to build firmware using your
-platform's build tool:
-
-1. To enable AutoTVM and AutoScheduling on your platform. Defining a Project
- API implementation allows TVM to tune models for peak performance on your
- platform.
-2. To enable engineers without firmware expertise to experiment with models on
- your platform. Defining a Project API implementation allows these engineers
- to leverage the standard TVM Python workflows to perform host-driven
- inference on your platform.
-3. Integration Testing. Defining a Project API implementation allows you to
- create Continuous Integration Tests which verify model correctness and
- performance on your platform.
-
-API Definition
---------------
-
-The full API is the ``abstractmethod`` defined on ``ProjectAPIHandler`` in
-`python/tvm/micro/project_api/server.py <https://github.com/apache/tvm/blob/main/python/tvm/micro/project_api/server.py>`_.
-Rather than duplicate the documentation here, we simply refer you to that class.
-
-How TVM uses Project API
-------------------------
-
-This section explains how the Project API should be used with TVM. Project API
-is defined around the *Project* as the buildable unit of firmware. TVM expects
-to be provided initially with a directory containing a *Template Project*, which
-together with a :ref:`Model Library Format <model_library_format>` file can be
-built into a runnable project.
-
-Inside the Template Directory is (typically) a Python script implementing the
-API server. TVM launches this script in a subprocess and sends commands to the
-server to perform each of the actions outlined above.
-
-The typical usage flow is as follows:
-
-1. Launch Project API server in Template Project.
-2. Verify the API server is version-compatible with TVM, plus read properties
- of the implementation, by sending ``server_info_query`` command.
-3. Generate a new project by sending command ``generate_project`` to create a
- new project. The arguments to this command is a Model Library Format and a
- non-existent directory which should be populated with the generated
- project. The Template Project API server should copy itself into the
- newly-generated project.
-4. Terminate the Template Project API server.
-5. Launch Project API server in Generated Project.
-6. Verify the API server is version-compatible with TVM, plus read properties
- of the implementation, by sending ``server_info_query`` command.
-7. Build and flash the projec by sending commands ``build`` and ``flash`` to the
- API server.
-8. Communicate with the target. Send command ``open_transport`` followed by
- commands ``write_transport`` and ``read_transport`` to write and read from
- e.g. a serial port attached to the target. Upon completion,
- ``close_transport`` is sent.
-9. Terminate Project API server.
-
-Disk Layout of the Project
---------------------------
-
-In the root directory of a project (template or generated), one of the following
-two files must exist:
-
-- ``microtvm_api_server.py`` - the suggested approach. Place a
- python3-compatible Python script in the root directory. TVM will execute this
- script in its own process using the same interpreter used to execute TVM.
-- ``microtvm_api_server.sh`` (on Windows, ``microtvm_api_server.bat``) -
- alternate approach. When a different Python interpreter is necessary, or
- when you want to implement the server in a different language, create this
- executable file. TVM will launch this file in a separate process.
-
-Aside from these two files, no other restrictions are made on the layout.
-
-Communication between TVM and Project API Server
-------------------------------------------------
-
-TVM communicates with the Project API server using `JSON-RPC 2.0
-<https://www.jsonrpc.org/specification>`_. TVM always launches API servers using
-the following command-line:
-
-``microtvm_api_server.py --read-fd <n> --write-fd <n>``
-
-Commands are sent from TVM to the server over the file descriptor given by
-``--read-fd`` and replies are received by TVM from the server over the file
-descriptor given by ``--write-fd``.
-
-Helpers for Implementing the API server in Python
--------------------------------------------------
-
-TVM provides helper utilities that make it easy to implement the server in Python.
-To implement the server in Python, create ``microtvm_api_server.py`` and add
-``from tvm.micro.project_api import server`` (or, copy this file into your template
-project--there are no dependencies--and import it there instead). Next, subclass
-``ProjectAPIHander``::
-
- class Handler(server.ProjectAPIHandler):
- def server_info_query(self, tvm_version):
- # Implement server_info_query
-
- def generate_project(self, model_library_format_path, standalone_crt_dir, project_dir, options):
- # Implement generate_project
-
- # ...
-
-Finally, invoke the helper ``main()``::
-
- if __name__ == "__main__":
- server.main(Handler())
-
-Using Project API from ``tvmc``
--------------------------------
-
-Each major Project API command is available through the ``tvmc micro``
-sub-command to make debugging interactions simple. Invoke ``tvmc micro --help``
-for more information.
diff --git a/docs/_sources/arch/model_library_format.rst.txt b/docs/_sources/arch/model_library_format.rst.txt
deleted file mode 100644
index 3ee6b9878f..0000000000
--- a/docs/_sources/arch/model_library_format.rst.txt
+++ /dev/null
@@ -1,171 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-.. _model_library_format:
-
-Model Library Format
-====================
-
-About Model Library Format
---------------------------
-
-TVM traditionally exports generated libraries as Dynamic Shared Objects (e.g. DLLs (Windows) or .so
-(linux)). Inferences can be performed using those libraries by loading them into an executable using
-``libtvm_runtime.so``. This process is very dependent on services provided by traditional OS.
-
-For deployment to unconventional platforms (e.g. those lacking traditional OS), TVM provides another
-output format, Model Library Format. Initially, the microTVM project is the primary use case for this
-format. Should it become useful in other use cases (and in particular, should it become possible to
-export BYOC artifacts in Model Library Format), it could be used as a general-purpose TVM export
-format. Model Library Format is a tarball containing a file for each piece of the TVM compiler
-output.
-
-What can be Exported?
----------------------
-
-At the time of writing, export is limited to full models built with ``tvm.relay.build``.
-
-Directory Layout
-----------------
-
-Model Library Format is contained within a tarball. All paths are relative to the root of the
-tarball:
-
-- ``/`` - Root of the tarball
-
- - ``codegen`` - Root directory for all generated device code
-
- - (see `codegen`_ section)
-
- - ``executor-config/`` - Configuration for the executor which drives model inference
-
- - ``graph/`` - Root directory containing configuration for the GraphExecutor
-
- - ``graph.json`` - GraphExecutor JSON configuration
-
- - ``metadata.json`` - Machine-parseable metadata for this model
-
- - ``parameters/`` - Root directory where simplified parameters are placed
-
- - ``<model_name>.params`` - Parameters for the model tvm.relay._save_params format
-
- - ``src/`` - Root directory for all source code consumed by TVM
-
- - ``relay.txt`` - Relay source code for the generated model
-
-Description of Sub-directories
-------------------------------
-
-.. _subdir_codegen:
-
-``codegen``
-^^^^^^^^^^^
-
-All TVM-generated code is placed in this directory. At the time of writing, there is 1 file per
-Module in the generated Module tree, though this restriction may change in the future. Files in
-this directory should have filenames of the form ``<target>/(lib|src)/<unique_name>.<format>``.
-
-These components are described below:
-
- * ``<target>`` - Identifies the TVM target on which the code should run. Currently, only ``host``
- is supported.
- * ``<unique_name>`` - A unique slug identifying this file. Currently ``lib<n>``, with ``<n>>`` an
- auto-incrementing integer.
- * ``<format>`` - Suffix identifying the filename format. Currently ``c`` or ``o``.
-
-An example directory tree for a CPU-only model is shown below:
-
-- ``codegen/`` - Codegen directory
-
- - ``host/`` - Generated code for ``target_host``
-
- - ``lib/`` - Generated binary object files
-
- - ``lib0.o`` - LLVM module (if ``llvm`` target is used)
- - ``lib1.o`` - LLVM CRT Metadata Module (if ``llvm`` target is used)
-
- - ``src/`` - Generated C source
-
- - ``lib0.c`` - C module (if ``c`` target is used)
- - ``lib1.c`` - C CRT Metadata module (if ``c`` target is used)
-
-``executor-config``
-^^^^^^^^^^^^^^^^^^^
-
-Contains machine-parsable configuration for executors which can drive model inference. Currently,
-only the GraphExecutor produces configuration for this directory, in ``graph/graph.json``. This
-file should be read in and the resulting string supplied to the ``GraphExecutor()`` constructor for
-parsing.
-
-``parameters``
-^^^^^^^^^^^^^^
-
-Contains machine-parseable parameters. A variety of formats may be provided, but at present, only
-the format produced by ``tvm.relay._save_params`` is supplied. When building with
-``tvm.relay.build``, the ``name`` parameter is considered to be the model name. A single file is
-created in this directory ``<model_name>.json``.
-
-``src``
-^^^^^^^
-
-Contains source code parsed by TVM. Currently, just the Relay source code is created in
-``src/relay.txt``.
-
-Metadata
---------
-
-Machine-parseable metadata is placed in a file ``metadata.json`` at the root of the tarball.
-Metadata is a dictionary with these keys:
-
-- ``export_datetime``: Timestamp when this Model Library Format was generated, in
- `strftime <https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior>`_
- format ``"%Y-%M-%d %H:%M:%SZ",``.
-- ``memory``: A summary of the memory usage of each generated function. Documented in
- `Memory Usage Summary`_.
-- ``model_name``: The name of this model (e.g. the ``name`` parameter supplied to
- ``tvm.relay.build``).
-- ``executors``: A list of executors supported by this model. Currently, this list is always
- ``["graph"]``.
-- ``target``: A dictionary mapping ``device_type`` (the underlying integer, as a string) to the
- sub-target which describes that relay backend used for that ``device_type``.
-- ``version``: A numeric version number that identifies the format used in this Model Library
- Format. This number is incremented when the metadata structure or on-disk structure changes.
- This document reflects version ``5``.
-
-Memory Usage Summary
-^^^^^^^^^^^^^^^^^^^^
-
-A dictionary with these sub-keys:
-
- - ``"main"``: ``list[MainFunctionWorkspaceUsage]``. A list summarizing memory usage for each
- workspace used by the main function and all sub-functions invoked.
- - ``"operator_functions"``: ``map[string, list[FunctionWorkspaceUsage]]``. Maps operator function
- name to a list summarizing memory usage for each workpace used by the function.
-
-A ``MainFunctionWorkspaceUsage`` is a dict with these keys:
-
-- ``"device"``: ``int``. The ``device_type`` associated with this workspace.
-- ``"workspace_size_bytes"``: ``int``. Number of bytes needed in this workspace by this function
- and all sub-functions invoked.
-- ``"constants_size_bytes"``: ``int``. Size of the constants used by the main function.
-- ``"io_size_bytes"``: ``int``. Sum of the sizes of the buffers used from this workspace by this
- function and sub-functions.
-
-A ``FunctionWorkspaceUsage`` is a dict with these keys:
-
-- ``"device"``: ``int``. The ``device_type`` associated with this workspace.
-- ``"workspace_size_bytes"``: ``int``. Number of bytes needed in this workspace by this function.
diff --git a/docs/_sources/arch/relay_intro.rst.txt b/docs/_sources/arch/relay_intro.rst.txt
deleted file mode 100644
index 87f68fcbce..0000000000
--- a/docs/_sources/arch/relay_intro.rst.txt
+++ /dev/null
@@ -1,206 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-.. _relay-dev-intro:
-
-Introduction to Relay IR
-========================
-This article introduces Relay IR -- the second generation of NNVM.
-We expect readers from two kinds of background -- those who have a programming language background and deep learning
-framework developers who are familiar with the computational graph representation.
-
-We briefly summarize the design goal here, and will touch upon these points in the later part of the article.
-
-- Support traditional data flow-style programming and transformations.
-- Support functional-style scoping, let-binding and making it a fully featured differentiable language.
-- Being able to allow the user to mix the two programming styles.
-
-Build a Computational Graph with Relay
---------------------------------------
-Traditional deep learning frameworks use computational graphs as their intermediate representation.
-A computational graph (or dataflow graph), is a directed acyclic graph (DAG) that represents the computation.
-Though dataflow graphs are limited in terms of the computations they are capable of expressing due to
-lacking control flow, their simplicity makes it easier to implement automatic differentiation and
-compile for heterogeneous execution environments (e.g., executing parts of the graph on specialized hardware).
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/relay/dataflow.png
- :align: center
-
-
-You can use Relay to build a computational (dataflow) graph. Specifically, the above code shows how to
-construct a simple two-node graph. You can find that the syntax of the example is not that different from existing
-computational graph IR like NNVMv1, with the only difference in terms of terminology:
-
-- Existing frameworks usually use graph and subgraph
-- Relay uses function e.g. -- ``fn (%x)``, to indicate the graph
-
-Each dataflow node is a CallNode in Relay. The Relay Python DSL allows you to construct a dataflow graph quickly.
-One thing we want to highlight in the above code -- is that we explicitly constructed an Add node with
-both input point to ``%1``. When a deep learning framework evaluates the above program, it will compute
-the nodes in topological order, and ``%1`` will only be computed once.
-While this fact is very natural to deep learning framework builders, it is something that might
-surprise a PL researcher in the first place. If we implement a simple visitor to print out the result and
-treat the result as nested Call expression, it becomes ``log(%x) + log(%x)``.
-
-Such ambiguity is caused by different interpretations of program semantics when there is a shared node in the DAG.
-In a normal functional programming IR, nested expressions are treated as expression trees, without considering the
-fact that the ``%1`` is actually reused twice in ``%2``.
-
-The Relay IR is mindful of this difference. Usually, deep learning framework users build the computational
-graph in this fashion, where a DAG node reuse often occurs. As a result, when we print out the Relay program in
-the text format, we print one CallNode per line and assign a temporary id ``(%1, %2)`` to each CallNode so each common
-node can be referenced in later parts of the program.
-
-Module: Support Multiple Functions (Graphs)
--------------------------------------------
-So far we have introduced how can we build a dataflow graph as a function. One might naturally ask: Can we support multiple
-functions and enable them to call each other? Relay allows grouping multiple functions together in a module; the code below
-shows an example of a function calling another function.
-
-.. code::
-
- def @muladd(%x, %y, %z) {
- %1 = mul(%x, %y)
- %2 = add(%1, %z)
- %2
- }
- def @myfunc(%x) {
- %1 = @muladd(%x, 1, 2)
- %2 = @muladd(%1, 2, 3)
- %2
- }
-
-The Module can be viewed as a ``Map<GlobalVar, Function>``. Here GlobalVar is just an id that is used to represent the functions
-in the module. ``@muladd`` and ``@myfunc`` are GlobalVars in the above example. When a CallNode is used to call another function,
-the corresponding GlobalVar is stored in the op field of the CallNode. It contains a level of indirection -- we need to look up
-body of the called function from the module using the corresponding GlobalVar. In this particular case, we could also directly
-store the reference to the Function as op in the CallNode. So, why do we need to introduce GlobalVar? The main reason is that
-GlobalVar decouples the definition/declaration and enables recursion and delayed declaration of the function.
-
-.. code ::
-
- def @myfunc(%x) {
- %1 = equal(%x, 1)
- if (%1) {
- %x
- } else {
- %2 = sub(%x, 1)
- %3 = @myfunc(%2)
- %4 = add(%3, %3)
- %4
- }
- }
-
-In the above example, ``@myfunc`` recursively calls itself. Using GlobalVar ``@myfunc`` to represent the function avoids
-the cyclic dependency in the data structure.
-At this point, we have introduced the basic concepts in Relay. Notably, Relay has the following improvements over NNVMv1:
-
-- Succinct text format that eases debugging of writing passes.
-- First-class support for subgraphs-functions, in a joint module, this enables further chance of joint optimizations such as inlining and calling convention specification.
-- Naive front-end language interop, for example, all the data structure can be visited in Python, which allows quick prototyping of optimizations in Python and mixing them with C++ code.
-
-
-Let Binding and Scopes
-----------------------
-
-So far, we have introduced how to build a computational graph in the good old way used in deep learning frameworks.
-This section will talk about a new important construct introduced by Relay -- let bindings.
-
-Let binding is used in every high-level programming language. In Relay, it is a data structure with three
-fields ``Let(var, value, body)``. When we evaluate a let expression, we first evaluate the value part, assign
-it to the var, then return the evaluated result in the body expression.
-
-You can use a sequence of let bindings to construct a logically equivalent program to a dataflow program.
-The code example below shows one program with two forms side by side.
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/relay/dataflow_vs_func.png
- :align: center
-
-
-The nested let binding is called A-normal form, and it is commonly used as IRs in functional programming languages.
-Now, please take a close look at the AST structure. While the two programs are semantically identical
-(so are their textual representations, except that A-normal form has let prefix), their AST structures are different.
-
-Since program optimizations take these AST data structures and transform them, the two different structures will
-affect the compiler code we are going to write. For example, if we want to detect a pattern ``add(log(x), y)``:
-
-- In the data-flow form, we can first access the add node, then directly look at its first argument to see if it is a log
-- In the A-normal form, we cannot directly do the check anymore, because the first input to add is ``%v1`` -- we will need to keep a map from variable to its bound values and look up that map, in order to know that ``%v1`` is a log.
-
-Different data structures will impact how you might write transformations, and we need to keep that in mind.
-So now, as a deep learning framework developer, you might ask, Why do we need let bindings?
-Your PL friends will always tell you that let is important -- as PL is a quite established field,
-there must be some wisdom behind that.
-
-Why We Might Need Let Binding
------------------------------
-One key usage of let binding is that it specifies the scope of computation. Let us take a look at the following example,
-which does not use let bindings.
-
-.. image:: https://raw.githubusercontent.com/tvmai/tvmai.github.io/main/images/relay/let_scope.png
- :align: center
-
-The problem comes when we try to decide where we should evaluate node ``%1``. In particular, while the text format seems
-to suggest that we should evaluate node ``%1`` outside the if scope, the AST(as shown in the picture) does not suggest so.
-Actually, a dataflow graph never defines its scope of the evaluation. This introduces some ambiguity in the semantics.
-
-This ambiguity becomes more interesting when we have closures. Consider the following program, which returns a closure.
-We don’t know where should we compute ``%1``; it can be either inside or outside the closure.
-
-.. code::
-
- fn (%x) {
- %1 = log(%x)
- %2 = fn(%y) {
- add(%y, %1)
- }
- %2
- }
-
-A let binding solves this problem, as the computation of the value happens at the let node. In both programs,
-if we change ``%1 = log(%x)`` to ``let %v1 = log(%x)``, we clearly specify the computation location to
-be outside of the if scope and closure. As you can see let-binding gives a more precise specification of the computation site
-and could be useful when we generate backend code (as such specification is in the IR).
-
-On the other hand, the dataflow form, which does not specify the scope of computation, does have its own advantages
--- namely, we don’t need to worry about where to put the let when we generate the code. The dataflow form also gives more freedom
-to the later passes to decide where to put the evaluation point. As a result, it might not be a bad idea to use data flow
-form of the program in the initial phases of optimizations when you find it is convenient.
-Many optimizations in Relay today are written to optimize dataflow programs.
-
-However, when we lower the IR to an actual runtime program, we need to be precise about the scope of computation.
-In particular, we want to explicitly specify where the scope of computation should happen when we are using
-sub-functions and closures. Let-binding can be used to solve this problem in later stage execution specific optimizations.
-
-
-Implication on IR Transformations
----------------------------------
-
-Hopefully, by now you are familiar with the two kinds of representations.
-Most functional programming languages do their analysis in A-normal form,
-where the analyzer does not need to be mindful that the expressions are DAGs.
-
-Relay choose to support both the dataflow form and let bindings. We believe that it is important to let the
-framework developer choose the representation they are familiar with.
-This does, however, have some implications on how we write passes:
-
-- If you come from a dataflow background and want to handle lets, keep a map of var to the expressions so you can perform lookup when encountering a var. This likely means a minimum change as we already need a map from expressions to transformed expressions anyway. Note that this will effectively remove all the lets in the program.
-- If you come from a PL background and like A-normal form, we will provide a dataflow to A-normal form pass.
-- For PL folks, when you are implementing something (like a dataflow-to-ANF transformation), be mindful that expressions can be DAGs, and this usually means that we should visit expressions with a ``Map<Expr, Result>`` and only compute the transformed result once, so the resulting expression keeps the common structure.
-
-There are additional advanced concepts such as symbolic shape inference, polymorphic functions
-that are not covered by this material; you are more than welcome to look at other materials.
diff --git a/docs/_sources/arch/relay_op_strategy.rst.txt b/docs/_sources/arch/relay_op_strategy.rst.txt
deleted file mode 100644
index dbac7c8218..0000000000
--- a/docs/_sources/arch/relay_op_strategy.rst.txt
+++ /dev/null
@@ -1,282 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-.. _relay-op-strategy:
-
-Relay Operator Strategy
-=======================
-
-In order to lower Relay operators to the implementations defined in TOPI
-library, a compute and schedule function need to be registered to each Relay
-operator. However, compute and schedule functions are usually specialized for
-each target, and further, even for the same target, we may have multiple
-algorithms and implementations available. To deal with the complexity, we
-introduce operator strategy to allow developers to define a flexible lowering
-strategy for each operator and target.
-
-
-Operator Strategy Design
-------------------------
-
-The basic element in operator strategy is an ``OpImplementation``. It includes
-the a pair of compute and schedule function, the name of the implementation,
-and a priority level (the use of priority level is explained in
-`Select Implementation from Op Strategy`_).
-
-The ``OpStrategy`` includes a list of ``OpSpecialization``. Each ``OpSpecialization``
-contains a list of ``OpImplementation`` associated with a ``SpecializedCondition``
-(see definition in ``include/tvm/te/schedule.h``). The ``SpecializedCondition``
-can be null, indicating the implementations are generally applicable;
-otherwise, the implementations are only considered when the specialized
-condition is satisfied. ``SpecializedCondition`` consists of a list
-of clauses defined in Tensor Expression in conjunctive normal form (CNF) and
-only supports conditions on tensor shapes.
-
-Last, a strategy function, or ``FTVMStrategy``, determines which pair(s) of
-compute and schedule functions should be used given a workload, and needs to be
-registered to each Relay operator. ``FTVMStrategy`` is a generic function (see
-``include/tvm/target/generic_func.h``), that can be overwritten for each
-target. The function signature is
-
-.. code:: c
-
- OpStrategy(const Attrs& attrs, const Array<Tensor>& inputs, const Type& out_type, const Target& target)
-
-that the function returns an ``OpStrategy`` given the op attributes, input
-tensors, output types, and target to compile to.
-
-
-Write A Strategy Function
--------------------------
-
-We recommend developers to write strategy function in Python as
-most TOPI compute and schedule functions are written in Python.
-In python, we provide ``OpStrategy`` class in ``pyton/tvm/relay/op/op.py``.
-It only has one API, which is to add an implementation to the strategy:
-
-.. code:: python
-
- def add_implementation(self, compute, schedule, name="default", plevel=10)
-
-
-We now take ``topk`` as an example to explain how to write the
-``FTVMStrategy`` function:
-
-.. code:: python
-
- # add to python/tvm/relay/op/strategy/generic.py
- @override_native_generic_func("topk_strategy")
- def topk_strategy(attrs, inputs, out_type, target):
- strategy = _op.OpStrategy()
- strategy.add_implementation(
- wrap_compute_topk(topi.topk),
- wrap_topi_schedule(topi.generic.schedule_topk),
- name="topk.generic")
- return strategy
-
- # add to each target file in python/tvm/relay/op/strategy, e.g., x86.py, cuda.py, etc.
- @topk_strategy.register(["cuda", "gpu"])
- def topk_strategy_cuda(attrs, inputs, out_type, target):
- strategy = _op.OpStrategy()
- strategy.add_implementation(
- wrap_compute_my_new_op(topi.cuda.topk),
- wrap_topi_schedule(topi.cuda.schedule_topk),
- name="topk.cuda")
- return strategy
-
-In this example, we use ``topi.cuda.topk`` and ``topi.cuda.schedule_topk``
-as the compute and schedule function for CUDA or GPU target, while use TOPI
-generic compute and schedule for the rest of targets.
-Note that we use two wrapper functions that wrap the topi
-compute and schedule to conform with the required function signature (
-see ``FTVMCompute`` and ``FTVMSchedule`` in ``include/tvm/relay/op_attr_types.h``).
-Usually we need to write a customized compute wrapper function for each operator
-to get different fields from op attributes.
-
-The example above shows a very basic strategy function that only
-adds one implementation in the strategy. But for many complicated operators,
-we may need to add multiple implementations that use different algorithms.
-For example, we can use both direct and winograd algorithm to
-compute a conv2d op. In order to achieve this, we can write the strategy function
-as follows:
-
-.. code:: python
-
- strategy.add_implementation(
- wrap_compute_conv2d(topi.cuda.conv2d_nchw),
- wrap_topi_schedule(topi.cuda.schedule_conv2d_nchw),
- name="conv2d_nchw.cuda",
- plevel=10)
-
- if winograd_condition:
- strategy.add_implementation(
- wrap_compute_conv2d(topi.cuda.conv2d_nchw_winograd),
- wrap_topi_schedule(topi.cuda.schedule_conv2d_nchw_winograd),
- name="conv2d_nchw_winograd.cuda",
- plevel=15)
-
-In this example, we add two implementations to the conv2d strategy where
-winograd algorithm is only added when ``winograd_condition`` is true.
-The implementation ``"conv2d_nchw_winograd.cuda"`` will be used to compile
-conv2d when ``winograd_condition`` is true as it has higher
-priority level (this could be changed if certain implementation is an AutoTVM
-template. See `Select Implementation from Op Strategy`_ for more
-details). Otherwise, ``"conv2d_nchw.cuda"`` is used.
-
-We can extend the example above to third party library implementation. For
-example, we can add the implementation that invokes kernel in the cblas
-library when cblas is included in the target.
-
-.. code:: python
-
- if "cblas" in target.libs:
- strategy.add_implementation(
- wrap_compute_dense(topi.x86.dense_cblas),
- wrap_topi_schedule(topi.x86.schedule_dense_cblas),
- name="dense_cblas.x86",
- plevel=15)
-
-
-Further, we can add implementation specialized for a certain range of shapes.
-The code below shows an example of dense strategy that adds an implementation
-that is specialized for ``m`` greater than 16. The main difference between
-hardcode python condition like examples above and specialized condition is that
-it allows TVM to generate multiple kernels when the input tensors have symbolic
-shapes. The compile engine will generate a dispatch function that invokes the
-specialized kernel when the corresponding condition is met; otherwise,
-invoke the kernel that has no associated specialized condition (``dense_common``
-in this example). This part is still work in progress. More details will be
-provided after it is done.
-
-.. code:: python
-
- def dense_strategy(attrs, inputs, out_type, target):
- m = inputs[0].shape[0]
- strategy = _op.OpStrategy()
- strategy.add_implementation(
- wrap_compute_dense(dense_compute1),
- wrap_topi_schedule(dense_schedule1),
- name="dense_common")
-
- with tvm.te.SpecializedCondition(m > 16):
- strategy.add_implementation(
- wrap_compute_dense(dense_compute2),
- wrap_topi_schedule(dense_schedule2),
- name="dense_for_large_m",
- plevel=15)
-
- return strategy
-
-
-Register Strategy Function to An Operator
------------------------------------------
-
-After we define the strategy function for an operator, we can now
-register the strategy function to this operator with
-
-.. code:: python
-
- register_strategy("topk", strategy.topk_strategy)
-
-However, it takes much effort to write a strategy function for an operator.
-Therefore, we provide two other methods for simpler operators.
-
-First, for operators that have injective, broadcast, or reduction pattern, we
-can call ``register_injective_schedule``, ``register_broadcast_schedule``, and
-``register_reduce_schedule`` repsectively. The schedule function for these
-patterns are already registered by each target and can be applied to these
-operators. We assume the compute function should be the same across all targets,
-and ``FTVMCompute`` needs to be registered to the op before invoking register
-schedule.
-
-.. code:: python
-
- register_broadcast_schedule("add")
-
-Second, for operators that doesn't have these common patterns mentioned before,
-but also have the same compute function for all targets, we can use
-``register_schedule`` API. It is easier to write ``FTVMSchedule`` function
-as we only need to provide which schedule function to use. The following
-code snippet shows ``FTVMSchedule`` function for pooling.
-
-.. code:: python
-
- # add to python/tvm/relay/op/strategy/generic.py
- @generic_func
- def schedule_pool(attrs, outs, target):
- with target:
- return topi.generic.schedule_pool(outs, attrs.layout)
-
- # add to each target file in python/tvm/relay/op/strategy, e.g., x86.py, cuda.py, etc.
- @schedule_pool.register("cpu")
- def schedule_pool_cpu(attrs, outs, target):
- ...
-
-After we created the ``FTVMSchedule`` for an operator, we can
-register the strategy using ``register_schedule``:
-
-.. code:: python
-
- register_schedule("nn.max_pool2d", strategy.schedule_pool)
-
-
-Register Strategies for A New Target
-------------------------------------
-
-There are two ways to register strategies for a new target. The more
-straightforward one is adding a new target file in the directory
-``python/tvm/relay/op/strategy``. You only need to customize the strategy for
-ops that have been implemented for this new target and reuse the generic
-strategies for the rest.
-
-Alternatively, you can also register the strategy for the new target outside the
-TVM python library. The following code snippet shows an example how to do
-so. You can find more examples in ``vta/python/vta/top/op.py``.
-
-.. code:: python
-
- @relay.op.strategy.conv2d_strategy.register("mytarget")
- def conv2d_strategy_mytarget(attrs, inputs, out_type, target):
- ...
-
-
-Select Implementation from Op Strategy
---------------------------------------
-
-During the compilation, Relay compile engine needs to determine which
-implementation to use for an operator when there are multiple. The selection
-policy works as follows.
-
-When the input tensors to an operator or a fused op all have constant shapes,
-the compile engine first finds the best implementation based on AutoTVM tuning
-logs. If there is no implementation that is an AutoTVM template or all AutoTVM
-templates have fallback configs, the implementation with highest priority level
-will then be chosen. Implementations with same priority level in this case leads
-to an undefined behavior, and any of them might be selected.
-
-The selection policy for ops with symbolic input shapes is still work in
-progress. Currently, if any input tensor has a symbolic shape, only the
-implementation with highest priority level will be used for this operator. This
-will be updated after the implementation finishes.
-
-For debug purpose, you can add the following lines before you compile the Relay
-model to learn which implementation is used for each operator.
-
-.. code:: python
-
- logging.getLogger("te_compiler").setLevel(logging.INFO)
- logging.getLogger("te_compiler").addHandler(logging.StreamHandler(sys.stdout))
diff --git a/docs/_sources/arch/virtual_machine.rst.txt b/docs/_sources/arch/virtual_machine.rst.txt
deleted file mode 100644
index c532392afe..0000000000
--- a/docs/_sources/arch/virtual_machine.rst.txt
+++ /dev/null
@@ -1,410 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-Putting the VM in TVM: The Relay Virtual Machine
-================================================
-
-Relay, a new program representation, has enabled the representation and optimization of
-a great breadth of machine learning programs.
-Unfortunately, by supporting a more expressive set of programs, we have
-introduced several new execution challenges.
-
-Relay's interpreter can execute the full language but has notable limitations
-that make it unsuited for production deployments. It is structured as an inefficient
-interpreter that performs AST traversal to execute the program. This approach is conceptually
-simple but inefficient, as the AST traversal heavily relies on indirection.
-
-There are further challenges in compiling dynamic code, such as dynamic scheduling and allocation,
-fully dynamic tensor shapes, and control flow. The interpreter offers simple solutions
-for these, but none is sufficiently compelling or optimized.
-
-The second execution mechanism is the existing graph executor. In order to target Relay
-programs to this, we compile a small subset of them to the old graph format and execute
-them on the runtime. Graph executor provides a fast execution experience but only for a very limited
-subset of Relay programs.
-
-An alternative but not-standard approach is Relay's ahead-of-time compiler,
-which compiles a Relay program into a shared library containing an ahead-of-time
-implementation. The ahead-of-time compiler provides compelling performance
-but is difficult to extend and instrument, which can only be done by modifying the
-code generation and optimization mechanisms.
-
-The Relay virtual machine is intended to be a framework that balances these competing
-approaches, providing a dynamic execution environment which can be extended, instrumented,
-and integrated with other approaches like ahead-of-time compilation via a flexible extension
-mechanism.
-
-The virtual machine is designed to strike a balance between performance and flexibility
-when deploying and executing Relay programs, without giving up the benefits of TVM.
-
-Virtual machine (VM) design is a well-studied area in programming languages and systems,
-and there have been various virtual machine designs for both full-fledged
-and embedded programing languages.
-Previous language VM designs have been heavily tailored to the execution profile of traditional programs.
-Traditional programs manipulate small scalar values and consist of a large number of low-level instructions.
-The sheer quantity of instructions requires instruction execution and dispatch to be extremely efficient.
-In the context of machine learning we manipulate primarily tensor values, using a (relatively)
-low number of high level instructions. ML programs' cost centers are expensive operator invocations,
-such as GEMM or convolution, over a large input. Due to the execution profile exhibited by ML programs,
-micro-optimizations present in scalar VMs are dramatically less important.
-
-TVM has provided strong support for vision models,
-but we want to grow to support a wider variety of models.
-The graph executor is able to utilize the fully static nature of the input graphs to perform
-aggressive optimization such as fully static allocation, and optimal memory reuse.
-When we introduce models which make use of control flow, recursion, dynamic shapes, and dynamic
-allocation, we must change how execution works. A virtual machine for Relay is a natural choice.
-
-The rest of this document provides a high-level overview of the Relay
-virtual machine design and its instruction set.
-
-Design
-------
-
-The VM's design is focused on simplicity without sacrificing performance.
-In order to accomplish this we have focused on designing a tensor VM rather than a scalar VM.
-
-In the tensor VM setting, we optimize for cheap “allocation” of objects (by trying to avoid real allocation),
-reuse of static fragments, and the ability to do dynamic shape (i.e jagged tensors).
-
-Instruction Set
-~~~~~~~~~~~~~~~
-
-The choices of an instruction set and instruction representation are the most critical design decisions for a VM.
-The current representation of the instructions is a tagged union containing the op-code and the data payload. An important design decision is the level of abstraction of the instructions (RISC vs. CISC) and how they take their data (fixed-width instruction encoding vs. variable-length encoding). The current version is closer to CISC, with complex instructions like AllocTensor, and is variable-length due to the inclusion of the shape as part of the instruction. The current instruction se [...]
-
-Ret
-^^^
-**Arguments**:
-::
-
- RegName dst
- RegName result
-
-Returns the object in register ``result`` to caller's register ``dst``.
-
-InvokePacked
-^^^^^^^^^^^^
-**Arguments**:
-::
-
- Index packed_index
- Index arity
- Index output_size
- RegName* packed_args
-
-Invoke the packed function denoted by ``packed_index``. The ``arity``
-and ``output_size`` are used to inform the VM how many inputs and
-outputs to expect. ``packed_args`` stores the list of argument registers. Note ``Index``
-is an alias of ``int64_t``, and it will be used in other instructions as well.
-
-AllocTensor
-^^^^^^^^^^^
-**Arguments**:
-::
-
- RegName dst
- RegName storage
- uint32_t ndim
- int64_t* shape
- DLDataType dtype
-
-Allocate a tensor value of using constant shape (stored in ``shape``) and ``dtype``
-from the given storage block, ``storage``. The result is saved to register ``dst``.
-
-AllocTensorReg
-^^^^^^^^^^^^^^
-**Arguments**:
-::
-
- RegName dst
- RegName storage
- RegName shape_register
- DLDataType dtype
-
-Allocate a tensor value of the appropriate shape (stored in ``shape_register``)
-and ``dtype`` from the given storage block (stored in ``storage``). The result is saved to register ``dst``.
-
-AllocStorage
-^^^^^^^^^^^^
-**Arguments**:
-::
-
- RegName dst
- RegName size
- RegName alignment
- DLDataType dtype_hint
-
-Allocate a storage block with the given ``size``, ``alignment`` and data type, ``dtype_hint``.
-The allocated storage block is stored in register ``dst``.
-
-AllocADT
-^^^^^^^^
-**Arguments**:
-::
-
- RegName dst
- Index tag
- Index num_fields
- RegName* datatype_fields
-
-Allocate a data type with the tag ``tag`` using the ``num_fields`` entries
-from registers ``datatype_fields``. The result is saved to register ``dst``.
-
-AllocClosure
-^^^^^^^^^^^^
-**Arguments**:
-::
-
- RegName dst
- Index clo_index
- Index num_freevar
- RegName* free_vars;
-
-Allocate a closure with the VMFunction at ``clo_index`` as
-its code, and the ``num_freevar`` entries from registers in
-``free_vars``. The result is saved to register ``dst``.
-
-GetField
-^^^^^^^^
-**Arguments**:
-::
-
- RegName dst
- RegName object
- Index field_index
-
-Get the field value with index ``field_index`` from ``object``. And saves the result to register ``dst``.
-
-If
-^^
-**Arguments**:
-::
-
- RegName test
- RegName target
- Index true_offset
- Index false_offset
-
-Check if the object at register ``test`` is equal to ``target``.
-If equal, relative jump by ``true_offset``, else relative
-jump by ``false_offset``.
-
-GetTag
-^^^^^^
-**Arguments**:
-::
-
- RegName object
- RegName dst
-
-Get the object tag for ADT object in register ``object``. And saves the reult to register ``dst``.
-
-Fatal
-^^^^^
-Fail the virtual machine execution.
-
-Goto
-^^^^
-**Arguments**:
-::
-
- Index pc_offset
-
-Relative unconditional jump by ``pc_offset``.
-
-Invoke
-^^^^^^
-**Arguments**:
-::
-
- Index func_index
-
-Invoke function at ``func_index``, consumes the number of arguments contained in the VMFunction's
-arity field.
-
-InvokeClosure
-^^^^^^^^^^^^^
-**Arguments**:
-::
-
- RegName closure
- Index num_closure_args
- RegName* closure_args
-
-Invokes ``closure``, consuming the number of arguments declared in the closure's VMFunction.
-
-LoadConst
-^^^^^^^^^
-**Arguments**:
-::
-
- RegName dst
- Index const_index
-
-Load the constant at ``const_index`` from the constant pool. The result is saved to register ``dst``.
-
-LoadConsti
-^^^^^^^^^^
-**Arguments**:
-::
-
- Index val
- RegName dst
-
-Load the constant integer ``val`` to register ``dst``. The result is a 0-rank tensor.
-
-Object Representation
-~~~~~~~~~~~~~~~~~~~~~
-We leverage the object protocol to represent the objects that are used by the
-VM.
-
-Currently, three types of objects, ``NDArray``, ``ADT``, and ``Closure`` objects, are used
-to represent tensor, tuple/list, and closure data, respectively. More details
-for each of them can be found at `include/tvm/runtime/ndarray.h`_,
-`include/tvm/runtime/vm/vm.h`_, and `include/tvm/runtime/container.h`_, respectively.
-
-.. _include/tvm/runtime/ndarray.h: https://github.com/apache/tvm/blob/main/include/tvm/runtime/ndarray.h
-
-.. _include/tvm/runtime/vm/vm.h: https://github.com/apache/tvm/blob/main/include/tvm/runtime/vm/vm.h
-
-.. _include/tvm/runtime/container.h: https://github.com/apache/tvm/blob/main/include/tvm/runtime/container.h
-
-Stack and State
-~~~~~~~~~~~~~~~
-
-The Relay VM maintains a stack frame, which contains information about how to resume the
-previous call. Registers are allocated in a continuous space (virtual register file) for each function.
-
-We keep track of a set of Relay functions we have called, a pointer into its bytecode, an offset into the byte code (known as the program counter).
-
-.. code-block:: c
-
- struct VirtualMachine {
- ...
- std::vector<VMFrame> frames;
- ...
- // Current function.
- size_t func_index;
- // Pointer into the current function's instructions.
- const Instruction* code;
- // Current program counter relative to the code pointer.
- size_t pc;
- ...
- };
-
-
-Dispatch Loop
-~~~~~~~~~~~~~
-A critical piece of a VM is the dispatch loop. The dispatch loop usually dominates the execution time of a
-virtual machine, but we have experimentally found this not to be the case for Relay. We have just implemented
-a simple ``switch``/``goto`` dispatch loop which dispatches based on instruction op code.
-
-This loop is implemented by ``VirtualMachine::Run()``.
-
-VM Compiler
-~~~~~~~~~~~
-
-An important part of this infrastructure is a compiler from Relay's full IR into a sequence of bytecode.
-The VM compiler transforms a ``tvm::relay::Module`` into a ``tvm::relay::vm::Executable``. The executable
-contains a set of compiled functions, the compiled functions are contained in ``tvm::relay::vm::Function``.
-The functions contain metadata about the function as well as its compiled bytecode. The emitted executable
-object then can be loaded and run by a ``tvm::relay::vm::VirtualMachine`` object. For full definitions of the
-data structures, please see `include/tvm/runtime/vm/executable.h`_ and `include/tvm/runtime/vm/vm.h`_.
-
-.. _include/tvm/runtime/vm/executable.h: https://github.com/apache/tvm/blob/main/include/tvm/runtime/vm/executable.h
-
-Optimizations
-~~~~~~~~~~~~~
-
-There are quite a few optimizations required by the VM compiler. Each of them
-is implemented as a pass which is managed by the Relay pass manager.
-
-Optimizations marked with `TODO` are not implemented yet.
-
-- A-Normal Form
-- Lambda Lift (see `src/relay/vm/lambda_lift.cc`_)
-- Inline Primitives (see `src/relay/vm/inline_primitives.cc`_)
-- Constant Pool Layout (see `src/relay/backend/vm/compiler.cc`_)
-- Tail Call Optimization (TODO)
-- Liveness Analysis (TODO)
-
-.. _src/relay/vm/lambda_lift.cc: https://github.com/apache/tvm/blob/main/src/relay/backend/vm/lambda_lift.cc
-
-.. _src/relay/vm/inline_primitives.cc: https://github.com/apache/tvm/blob/main/src/relay/backend/vm/inline_primitives.cc
-
-.. _src/relay/backend/vm/compiler.cc: https://github.com/apache/tvm/blob/main/src/relay/backend/vm/compiler.cc
-
-Serialization
-~~~~~~~~~~~~~
-
-Serializing and deserializing the executable generated by the Relay VM compiler is a must as
-we may want to save the model to the disk and perform inference later. Previously, Relay has produced
-a serialized form in a json file for the graph executor. However, the same format is not directly
-applicable to the VM as it emits bytecode instead of graph-style programs.
-Serialization of an executable essentially needs to handle both model specific
-(i.e. weights and kernels) and VM related (i.e. bytecode and global function names) data.
-
-For kernels, we can conveniently leverage existing TVM infra to save and load
-the compiled library module. Here we only focus on serializing other several
-components in a binary format that is organized with the following sections in order.
-
-- Global section. This section contains the globals (function names) used by the virtual machine.
-
-- Constant section. This section is used to store the constant pool (i.e. weights of the model)
- for a virtual machine.
-
-- Primitive name section. This section is introduced to accommodate the list of primitive
- operator names that will be invoked by the virtual machine, i.e. the names
- starting with ``fused_``. The primitive names are used as symbols to look up
- function pointers in the compiled kernel library.
-
-- Code section. The VM functions, including bytecode, are sitting in this section. The dispatching
- loop iterates through this section to fetch instructions for execution.
-
-Hence, unlike the graph executor artifact that contains weight (.params), graph json (.json),
-and compiled kernel library (.so), the serialized executable artifact is composed of the Relay
-object file (.ro) and the compiled kernel library (.so).
-
-A ``save`` function is implemented to store the executable to the disk and
-serialize it into the above format. Meanwhile, a ``load_exec`` function is used to
-load the serialized kernel binary and executable related binary code, which will be again used to
-instantiate a VM object. Please refer to the `test_vm_serialization.py`_ file for more
-examples.
-
-.. _test_vm_serialization.py: https://github.com/apache/tvm/blob/main/tests/python/relay/test_vm_serialization.py
-
-Unresolved Questions
-~~~~~~~~~~~~~~~~~~~~
-
-How do we handle dynamic shapes?
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Dynamic shape support is ongoing work in TVM as we upgrade Relay, TVM's compiler. For the most recent updates on
-dynamic shape support, we recommend following updates in TVM's Discuss forum (https://discuss.tvm.apache.org/).
-
-How can we modify the VM to support JIT compilation of certain code paths?
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-In the code generation space there are still many tradeoffs to be analyzed and the VM is designed
-to be very flexible so we can modify it for future experiments.
-
-How do we support heterogenous execution?
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Heterogenous execution should work out of the box assuming we have annotated the appropriate device copies.
-In order to do this properly we need to run the device annotation and copying passes.
diff --git a/docs/_sources/deep_dive/relax/index.rst.txt b/docs/_sources/deep_dive/relax/index.rst.txt
index f891eb2793..2b7c4ea599 100644
--- a/docs/_sources/deep_dive/relax/index.rst.txt
+++ b/docs/_sources/deep_dive/relax/index.rst.txt
@@ -15,7 +15,7 @@
specific language governing permissions and limitations
under the License.
-.. _relax:
+.. _relax-deep-dive:
Relax
=====
diff --git a/docs/_sources/deep_dive/relax/tutorials/sg_execution_times.rst.txt b/docs/_sources/deep_dive/relax/tutorials/sg_execution_times.rst.txt
index 923e2eb63e..576c8a4fc5 100644
--- a/docs/_sources/deep_dive/relax/tutorials/sg_execution_times.rst.txt
+++ b/docs/_sources/deep_dive/relax/tutorials/sg_execution_times.rst.txt
@@ -5,10 +5,10 @@
Computation times
=================
-**00:00.249** total execution time for **deep_dive_relax_tutorials** files:
+**00:00.238** total execution time for **deep_dive_relax_tutorials** files:
+-------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_deep_dive_relax_tutorials_relax_creation.py` (``relax_creation.py``) | 00:00.166 | 0.0 MB |
+| :ref:`sphx_glr_deep_dive_relax_tutorials_relax_creation.py` (``relax_creation.py``) | 00:00.157 | 0.0 MB |
+-------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_deep_dive_relax_tutorials_relax_transformation.py` (``relax_transformation.py``) | 00:00.083 | 0.0 MB |
+| :ref:`sphx_glr_deep_dive_relax_tutorials_relax_transformation.py` (``relax_transformation.py``) | 00:00.081 | 0.0 MB |
+-------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/deep_dive/tensor_ir/index.rst.txt b/docs/_sources/deep_dive/tensor_ir/index.rst.txt
index 46bed7c423..66e153ec01 100644
--- a/docs/_sources/deep_dive/tensor_ir/index.rst.txt
+++ b/docs/_sources/deep_dive/tensor_ir/index.rst.txt
@@ -15,7 +15,7 @@
specific language governing permissions and limitations
under the License.
-.. _tensor-ir:
+.. _tensor-ir-deep-dive:
TensorIR
========
diff --git a/docs/_sources/deep_dive/tensor_ir/tutorials/sg_execution_times.rst.txt b/docs/_sources/deep_dive/tensor_ir/tutorials/sg_execution_times.rst.txt
index 5ad03e9383..8edc9cdb35 100644
--- a/docs/_sources/deep_dive/tensor_ir/tutorials/sg_execution_times.rst.txt
+++ b/docs/_sources/deep_dive/tensor_ir/tutorials/sg_execution_times.rst.txt
@@ -5,10 +5,10 @@
Computation times
=================
-**00:00.542** total execution time for **deep_dive_tensor_ir_tutorials** files:
+**00:00.536** total execution time for **deep_dive_tensor_ir_tutorials** files:
+-------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_deep_dive_tensor_ir_tutorials_tir_transformation.py` (``tir_transformation.py``) | 00:00.320 | 0.0 MB |
+-------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_deep_dive_tensor_ir_tutorials_tir_creation.py` (``tir_creation.py``) | 00:00.222 | 0.0 MB |
+| :ref:`sphx_glr_deep_dive_tensor_ir_tutorials_tir_creation.py` (``tir_creation.py``) | 00:00.216 | 0.0 MB |
+-------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/deep_dive/tensor_ir/tutorials/tir_transformation.rst.txt b/docs/_sources/deep_dive/tensor_ir/tutorials/tir_transformation.rst.txt
index 6f575b914e..1bf1f08f91 100644
--- a/docs/_sources/deep_dive/tensor_ir/tutorials/tir_transformation.rst.txt
+++ b/docs/_sources/deep_dive/tensor_ir/tutorials/tir_transformation.rst.txt
@@ -123,7 +123,7 @@ original implementation.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 2.2330 2.2330 2.2330 2.2330 0.0000
+ 2.2499 2.2499 2.2499 2.2499 0.0000
@@ -295,7 +295,7 @@ action involves reordering these two loops.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 0.8525 0.8525 0.8525 0.8525 0.0000
+ 0.9042 0.9042 0.9042 0.9042 0.0000
@@ -423,7 +423,7 @@ from the reduction update via the **decompose_reduction** primitive.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 0.3212 0.3212 0.3212 0.3212 0.0000
+ 0.3216 0.3216 0.3216 0.3216 0.0000
diff --git a/docs/_sources/dev/tutorial/codebase_walkthrough.rst.txt b/docs/_sources/dev/tutorial/codebase_walkthrough.rst.txt
index 726e253057..a349b69f7b 100644
--- a/docs/_sources/dev/tutorial/codebase_walkthrough.rst.txt
+++ b/docs/_sources/dev/tutorial/codebase_walkthrough.rst.txt
@@ -124,7 +124,7 @@ Lowering is done by ``tvm.lower()`` function, defined in ``python/tvm/build_modu
stmt = schedule.ScheduleOps(sch, bounds)
...
-Bound inference is the process where all loop bounds and sizes of intermediate buffers are inferred. If you target the CUDA backend and you use shared memory, its required minimum size is automatically determined here. Bound inference is implemented in ``src/te/schedule/bound.cc``, ``src/te/schedule/graph.cc`` and ``src/te/schedule/message_passing.cc``. For more information on how bound inference works, see :ref:`dev-InferBound-Pass`.
+Bound inference is the process where all loop bounds and sizes of intermediate buffers are inferred. If you target the CUDA backend and you use shared memory, its required minimum size is automatically determined here. Bound inference is implemented in ``src/te/schedule/bound.cc``, ``src/te/schedule/graph.cc`` and ``src/te/schedule/message_passing.cc``.
``stmt``, which is the output of ``ScheduleOps()``, represents an initial loop nest structure. If you have applied ``reorder`` or ``split`` primitives to your schedule, then the initial loop nest already reflects those changes. ``ScheduleOps()`` is defined in ``src/te/schedule/schedule_ops.cc``.
diff --git a/docs/_sources/get_started/tutorials/sg_execution_times.rst.txt b/docs/_sources/get_started/tutorials/sg_execution_times.rst.txt
index 21f1394e52..92013e9a7e 100644
--- a/docs/_sources/get_started/tutorials/sg_execution_times.rst.txt
+++ b/docs/_sources/get_started/tutorials/sg_execution_times.rst.txt
@@ -5,10 +5,10 @@
Computation times
=================
-**00:02.216** total execution time for **get_started_tutorials** files:
+**00:02.296** total execution time for **get_started_tutorials** files:
+---------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_get_started_tutorials_ir_module.py` (``ir_module.py``) | 00:02.014 | 0.0 MB |
+| :ref:`sphx_glr_get_started_tutorials_ir_module.py` (``ir_module.py``) | 00:02.100 | 0.0 MB |
+---------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_get_started_tutorials_quick_start.py` (``quick_start.py``) | 00:00.202 | 0.0 MB |
+| :ref:`sphx_glr_get_started_tutorials_quick_start.py` (``quick_start.py``) | 00:00.196 | 0.0 MB |
+---------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/compile_models/from_darknet.rst.txt b/docs/_sources/how_to/compile_models/from_darknet.rst.txt
index 8fe6c3854b..a4ba58c5c0 100644
--- a/docs/_sources/how_to/compile_models/from_darknet.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_darknet.rst.txt
@@ -318,7 +318,7 @@ The process is no different from other examples.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 40.080 seconds)
+ **Total running time of the script:** ( 1 minutes 35.995 seconds)
.. _sphx_glr_download_how_to_compile_models_from_darknet.py:
diff --git a/docs/_sources/how_to/compile_models/from_keras.rst.txt b/docs/_sources/how_to/compile_models/from_keras.rst.txt
index 9fb02a3a7f..357885aa8a 100644
--- a/docs/_sources/how_to/compile_models/from_keras.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_keras.rst.txt
@@ -232,7 +232,7 @@ Look up prediction top 1 index in 1000 class synset.
.. code-block:: none
Relay top-1 id: 285, class name: Egyptian cat
- 1/1 [==============================] - ETA: 0s 1/1 [==============================] - 1s 868ms/step
+ 1/1 [==============================] - ETA: 0s 1/1 [==============================] - 1s 898ms/step
Keras top-1 id: 285, class name: Egyptian cat
diff --git a/docs/_sources/how_to/compile_models/from_oneflow.rst.txt b/docs/_sources/how_to/compile_models/from_oneflow.rst.txt
index b1521d53b6..393328080c 100644
--- a/docs/_sources/how_to/compile_models/from_oneflow.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_oneflow.rst.txt
@@ -121,7 +121,7 @@ Load a pretrained OneFlow model and save model
.. code-block:: none
Downloading: "https://oneflow-public.oss-cn-beijing.aliyuncs.com/model_zoo/flowvision/classification/ResNet/resnet18.zip" to /workspace/.oneflow/flowvision_cache/resnet18.zip
- 0%| | 0.00/41.5M [00:00<?, ?B/s] 15%|#5 | 6.33M/41.5M [00:00<00:02, 16.0MB/s] 19%|#9 | 7.99M/41.5M [00:00<00:02, 15.2MB/s] 35%|###4 | 14.3M/41.5M [00:00<00:01, 22.2MB/s] 41%|#### | 17.0M/41.5M [00:00<00:01, 23.4MB/s] 54%|#####3 | 22.3M/41.5M [00:01<00:00, 26.6MB/s] 60%|###### | 24.9M/41.5M [00:01<00:00, 24.2MB/s] 77%|#######7 | 32.0M/41.5M [00:01<00:00, 33.9MB/s] 92%|#########2| 38.3M/41.5M [00:01<00:00, 21.9MB/s] 100%|##########| 41.5M/41.5M [00:01<00:00, 23.9MB/s]
+ 0%| | 0.00/41.5M [00:00<?, ?B/s] 15%|#5 | 6.33M/41.5M [00:00<00:00, 46.2MB/s] 26%|##5 | 10.7M/41.5M [00:00<00:00, 39.2MB/s] 35%|###4 | 14.5M/41.5M [00:00<00:01, 27.5MB/s] 42%|####1 | 17.3M/41.5M [00:00<00:00, 25.8MB/s] 54%|#####3 | 22.3M/41.5M [00:00<00:00, 29.9MB/s] 61%|###### | 25.3M/41.5M [00:00<00:00, 23.3MB/s] 79%|#######8 | 32.7M/41.5M [00:01<00:00, 35.5MB/s] 92%|#########2| 38.3M/41.5M [00:01<00:00, 40.0MB/s] 100%|##########| 41.5M/41.5M [00:01<00:00, 34.1MB/s]
diff --git a/docs/_sources/how_to/compile_models/from_paddle.rst.txt b/docs/_sources/how_to/compile_models/from_paddle.rst.txt
index 3c0582d608..4d6d778634 100644
--- a/docs/_sources/how_to/compile_models/from_paddle.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_paddle.rst.txt
@@ -209,7 +209,7 @@ Look up prediction top 1 index in 1000 class synset.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 7.479 seconds)
+ **Total running time of the script:** ( 1 minutes 3.792 seconds)
.. _sphx_glr_download_how_to_compile_models_from_paddle.py:
diff --git a/docs/_sources/how_to/compile_models/from_pytorch.rst.txt b/docs/_sources/how_to/compile_models/from_pytorch.rst.txt
index 7764a96dda..c07d9d887c 100644
--- a/docs/_sources/how_to/compile_models/from_pytorch.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_pytorch.rst.txt
@@ -101,7 +101,7 @@ Load a pretrained PyTorch model
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/resnet18-f37072fd.pth" to /workspace/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth
- 0%| | 0.00/44.7M [00:00<?, ?B/s] 14%|#3 | 6.25M/44.7M [00:00<00:00, 63.1MB/s] 28%|##7 | 12.4M/44.7M [00:00<00:00, 45.2MB/s] 38%|###8 | 17.0M/44.7M [00:00<00:00, 45.5MB/s] 48%|####8 | 21.5M/44.7M [00:00<00:00, 39.5MB/s] 58%|#####8 | 26.0M/44.7M [00:00<00:00, 41.5MB/s] 72%|#######2 | 32.2M/44.7M [00:00<00:00, 47.7MB/s] 83%|########2 | 37.0M/44.7M [00:00<00:00, 47.1MB/s] 96%|#########6| 43.0M/44.7M [00:00<00:00, 51.3MB/s] 100%|##########| 44.7M/44.7M [00:00<00:00, 48.1MB/s]
+ 0%| | 0.00/44.7M [00:00<?, ?B/s] 31%|###1 | 14.0M/44.7M [00:00<00:00, 146MB/s] 73%|#######3 | 32.6M/44.7M [00:00<00:00, 175MB/s] 100%|##########| 44.7M/44.7M [00:00<00:00, 178MB/s]
diff --git a/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt b/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt
index c85ac9816c..e3aff37a24 100644
--- a/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt
@@ -424,7 +424,7 @@ Run the corresponding model on tensorflow
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 30.735 seconds)
+ **Total running time of the script:** ( 1 minutes 30.268 seconds)
.. _sphx_glr_download_how_to_compile_models_from_tensorflow.py:
diff --git a/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt b/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt
index 22c51f23d8..0b8f540bf3 100644
--- a/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt
@@ -5,24 +5,24 @@
Computation times
=================
-**06:55.350** total execution time for **how_to_compile_models** files:
+**06:42.396** total execution time for **how_to_compile_models** files:
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_darknet.py` (``from_darknet.py``) | 01:40.080 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_darknet.py` (``from_darknet.py``) | 01:35.995 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_tensorflow.py` (``from_tensorflow.py``) | 01:30.735 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_tensorflow.py` (``from_tensorflow.py``) | 01:30.268 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_paddle.py` (``from_paddle.py``) | 01:07.479 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_paddle.py` (``from_paddle.py``) | 01:03.792 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_oneflow.py` (``from_oneflow.py``) | 00:45.754 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_oneflow.py` (``from_oneflow.py``) | 00:43.812 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_coreml.py` (``from_coreml.py``) | 00:39.800 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_coreml.py` (``from_coreml.py``) | 00:40.016 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_pytorch.py` (``from_pytorch.py``) | 00:27.991 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_pytorch.py` (``from_pytorch.py``) | 00:27.622 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_keras.py` (``from_keras.py``) | 00:27.659 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_keras.py` (``from_keras.py``) | 00:26.096 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_tflite.py` (``from_tflite.py``) | 00:13.039 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_tflite.py` (``from_tflite.py``) | 00:11.995 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_onnx.py` (``from_onnx.py``) | 00:02.814 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_onnx.py` (``from_onnx.py``) | 00:02.800 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt
index 0e98a317fd..77127b11d7 100644
--- a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt
@@ -664,7 +664,7 @@ well as provides information about the model's performance
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 4083.5226 4123.6250 4156.7913 3997.0490 60.6314
+ 3990.3151 3989.4777 3996.4851 3986.9155 2.5642
@@ -672,7 +672,7 @@ well as provides information about the model's performance
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 21.212 seconds)
+ **Total running time of the script:** ( 1 minutes 19.022 seconds)
.. _sphx_glr_download_how_to_deploy_models_deploy_model_on_adreno.py:
diff --git a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt
index 2153d2a1db..98a16c83a5 100644
--- a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt
@@ -127,7 +127,7 @@ Make a Keras Resnet50 Model
.. code-block:: none
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/resnet/resnet50_weights_tf_dim_ordering_tf_kernels.h5
- 8192/102967424 [..............................] - ETA: 0s 6635520/102967424 [>.............................] - ETA: 2s 8380416/102967424 [=>............................] - ETA: 3s 16769024/102967424 [===>..........................] - ETA: 2s 23412736/102967424 [=====>........................] - ETA: 2s 25157632/102967424 [======>.......................] - ETA: 2s 33546240/102967424 [========>.....................] - ETA: 2s 40189952/102967424 [==========>...................] - ETA: 2s
41934848/102967424 [===========>..................] - ETA: 2s 48578560/102967424 [=============>................] - ETA: 1s 50323456/102967424 [=============>................] - ETA: 1s 56967168/102967424 [===============>..............] - ETA: 1s 58712064/102967424 [================>.............] - ETA: 1s 65355776/102967424 [==================>...........] - ETA: 1s 67100672/102967424 [==================>...........] - ETA: 1s 69296128/102967424 [===================>..........] -
ETA: 1s 73744384/102967424 [====================>.........] - ETA: 1s 75489280/102967424 [====================>.........] - ETA: 0s 82124800/102967424 [======================>.......] - ETA: 0s 83877888/102967424 [=======================>......] - ETA: 0s 90521600/102967424 [=========================>....] - ETA: 0s 92266496/102967424 [=========================>....] - ETA: 0s 98910208/102967424 [===========================>..] - ETA: 0s 100646912/102967424
[============================>.] - ETA: 0s 102850560/102967424 [============================>.] - ETA: 0s 102967424/102967424 [==============================] - 4s 0us/step
+ 8192/102967424 [..............................] - ETA: 0s 6635520/102967424 [>.............................] - ETA: 4s 8380416/102967424 [=>............................] - ETA: 4s 15024128/102967424 [===>..........................] - ETA: 3s 16769024/102967424 [===>..........................] - ETA: 3s 25157632/102967424 [======>.......................] - ETA: 2s 27680768/102967424 [=======>......................] - ETA: 2s 33546240/102967424 [========>.....................] - ETA: 1s
35741696/102967424 [=========>....................] - ETA: 1s 40189952/102967424 [==========>...................] - ETA: 2s 41934848/102967424 [===========>..................] - ETA: 1s 44130304/102967424 [===========>..................] - ETA: 1s 50323456/102967424 [=============>................] - ETA: 1s 58712064/102967424 [================>.............] - ETA: 1s 65355776/102967424 [==================>...........] - ETA: 1s 67100672/102967424 [==================>...........] -
ETA: 1s 69296128/102967424 [===================>..........] - ETA: 0s 73744384/102967424 [====================>.........] - ETA: 0s 75489280/102967424 [====================>.........] - ETA: 0s 80273408/102967424 [======================>.......] - ETA: 0s 82124800/102967424 [======================>.......] - ETA: 0s 83877888/102967424 [=======================>......] - ETA: 0s 90521600/102967424 [=========================>....] - ETA: 0s 92266496/102967424
[=========================>....] - ETA: 0s 98910208/102967424 [===========================>..] - ETA: 0s 101138432/102967424 [============================>.] - ETA: 0s 102850560/102967424 [============================>.] - ETA: 0s 102967424/102967424 [==============================] - 3s 0us/step
diff --git a/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt b/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt
index 8f693e2a02..c97fef7aa4 100644
--- a/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt
@@ -437,7 +437,7 @@ Execute on TVM
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 15.4221 15.4369 15.7172 15.2183 0.1370
+ 13.8390 13.7967 14.1738 13.7054 0.1546
diff --git a/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt b/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt
index 98fb847183..45461f0aca 100644
--- a/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt
@@ -130,7 +130,7 @@ Load pre-trained maskrcnn from torchvision and do tracing
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=MaskRCNN_ResNet50_FPN_Weights.COCO_V1`. You can also use `weights=MaskRCNN_ResNet50_FPN_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth" to /workspace/.cache/torch/hub/checkpoints/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth
- 0%| | 0.00/170M [00:00<?, ?B/s] 5%|4 | 8.12M/170M [00:00<00:01, 84.8MB/s] 11%|#1 | 19.0M/170M [00:00<00:01, 101MB/s] 21%|##1 | 35.9M/170M [00:00<00:01, 135MB/s] 31%|###1 | 52.8M/170M [00:00<00:00, 152MB/s] 41%|#### | 69.6M/170M [00:00<00:00, 160MB/s] 51%|#####1 | 86.6M/170M [00:00<00:00, 166MB/s] 62%|######1 | 104M/170M [00:00<00:00, 173MB/s] 72%|#######2 | 123M/170M [00:00<00:00, 180MB/s] 84%|########3 | 142M/170M [00:00<00:00, 186MB/s] 95%|#########4| 161M/170M [00:01<00:00, 190MB/s] 100%|##########| 170M/170M [00:01<00:00, 170MB/s]
+ 0%| | 0.00/170M [00:00<?, ?B/s] 9%|8 | 14.8M/170M [00:00<00:01, 154MB/s] 20%|#9 | 33.8M/170M [00:00<00:00, 180MB/s] 31%|### | 52.6M/170M [00:00<00:00, 188MB/s] 42%|####2 | 71.4M/170M [00:00<00:00, 191MB/s] 53%|#####3 | 90.2M/170M [00:00<00:00, 193MB/s] 64%|######4 | 109M/170M [00:00<00:00, 195MB/s] 75%|#######5 | 128M/170M [00:00<00:00, 195MB/s] 87%|########6 | 147M/170M [00:00<00:00, 196MB/s] 98%|#########7| 166M/170M [00:00<00:00, 197MB/s] 100%|##########| 170M/170M [00:00<00:00, 193MB/s]
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/detection/generalized_rcnn.py:75: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
for img in images:
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/detection/transform.py:122: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
@@ -330,7 +330,7 @@ Get boxes with score larger than 0.9
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 3 minutes 48.993 seconds)
+ **Total running time of the script:** ( 3 minutes 28.198 seconds)
.. _sphx_glr_download_how_to_deploy_models_deploy_object_detection_pytorch.py:
diff --git a/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt b/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt
index 93859d7158..58b740541a 100644
--- a/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt
@@ -227,7 +227,7 @@ training. Other models require a full post training calibration.
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=MobileNet_V2_Weights.IMAGENET1K_V1`. You can also use `weights=MobileNet_V2_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/mobilenet_v2-b0353104.pth" to /workspace/.cache/torch/hub/checkpoints/mobilenet_v2-b0353104.pth
- 0%| | 0.00/13.6M [00:00<?, ?B/s] 39%|###8 | 5.25M/13.6M [00:00<00:00, 50.4MB/s] 87%|########6 | 11.8M/13.6M [00:00<00:00, 60.0MB/s] 100%|##########| 13.6M/13.6M [00:00<00:00, 59.5MB/s]
+ 0%| | 0.00/13.6M [00:00<?, ?B/s] 100%|##########| 13.6M/13.6M [00:00<00:00, 153MB/s]
@@ -404,7 +404,7 @@ Here we give an example of how to measure performance of TVM compiled models.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 85.9805 85.9943 86.7402 85.6435 0.2291
+ 86.0503 86.0264 87.2294 85.7224 0.1936
@@ -452,7 +452,7 @@ TODO
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 30.124 seconds)
+ **Total running time of the script:** ( 1 minutes 30.928 seconds)
.. _sphx_glr_download_how_to_deploy_models_deploy_prequantized.py:
diff --git a/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt b/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt
index 7f7e5d3d80..da5bea393a 100644
--- a/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt
@@ -423,7 +423,7 @@ Here we give an example of how to measure performance of TVM compiled models.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 104.7561 103.9849 108.4503 103.4032 1.3820
+ 106.3776 106.1667 113.0510 105.3312 1.0507
diff --git a/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt b/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt
index 48cd92cbef..0553362000 100644
--- a/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt
@@ -5,24 +5,24 @@
Computation times
=================
-**10:28.240** total execution time for **how_to_deploy_models** files:
+**09:52.672** total execution time for **how_to_deploy_models** files:
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_object_detection_pytorch.py` (``deploy_object_detection_pytorch.py``) | 03:48.993 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_object_detection_pytorch.py` (``deploy_object_detection_pytorch.py``) | 03:28.198 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized.py` (``deploy_prequantized.py``) | 01:30.124 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized.py` (``deploy_prequantized.py``) | 01:30.928 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno.py` (``deploy_model_on_adreno.py``) | 01:21.212 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno.py` (``deploy_model_on_adreno.py``) | 01:19.022 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_android.py` (``deploy_model_on_android.py``) | 00:56.970 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_android.py` (``deploy_model_on_android.py``) | 00:52.823 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno_tvmc.py` (``deploy_model_on_adreno_tvmc.py``) | 00:54.164 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized_tflite.py` (``deploy_prequantized_tflite.py``) | 00:51.977 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized_tflite.py` (``deploy_prequantized_tflite.py``) | 00:51.102 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno_tvmc.py` (``deploy_model_on_adreno_tvmc.py``) | 00:49.157 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_nano.py` (``deploy_model_on_nano.py``) | 00:32.910 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_nano.py` (``deploy_model_on_nano.py``) | 00:30.387 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_rasp.py` (``deploy_model_on_rasp.py``) | 00:32.758 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_rasp.py` (``deploy_model_on_rasp.py``) | 00:30.172 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_deploy_models_deploy_sparse.py` (``deploy_sparse.py``) | 00:00.007 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt b/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt
index 7595bc8ace..d6c9eef6c7 100644
--- a/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt
@@ -5,12 +5,12 @@
Computation times
=================
-**00:03.889** total execution time for **how_to_extend_tvm** files:
+**00:04.047** total execution time for **how_to_extend_tvm** files:
+-------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_extend_tvm_use_pass_instrument.py` (``use_pass_instrument.py``) | 00:02.692 | 0.0 MB |
+| :ref:`sphx_glr_how_to_extend_tvm_use_pass_instrument.py` (``use_pass_instrument.py``) | 00:02.817 | 0.0 MB |
+-------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_extend_tvm_use_pass_infra.py` (``use_pass_infra.py``) | 00:01.189 | 0.0 MB |
+| :ref:`sphx_glr_how_to_extend_tvm_use_pass_infra.py` (``use_pass_infra.py``) | 00:01.222 | 0.0 MB |
+-------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_extend_tvm_low_level_custom_pass.py` (``low_level_custom_pass.py``) | 00:00.007 | 0.0 MB |
+| :ref:`sphx_glr_how_to_extend_tvm_low_level_custom_pass.py` (``low_level_custom_pass.py``) | 00:00.008 | 0.0 MB |
+-------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt b/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt
index 3c93bb4ba8..eeaf23a4cf 100644
--- a/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt
+++ b/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt
@@ -220,10 +220,10 @@ profile the execution time of each passes.
.. code-block:: none
Printing results of timing profile...
- InferType: 24425us [24425us] (48.62%; 48.62%)
- FoldScaleAxis: 25809us [8us] (51.38%; 51.38%)
- FoldConstant: 25801us [1770us] (51.36%; 99.97%)
- InferType: 24031us [24031us] (47.84%; 93.14%)
+ InferType: 24657us [24657us] (48.32%; 48.32%)
+ FoldScaleAxis: 26374us [11us] (51.68%; 51.68%)
+ FoldConstant: 26363us [1806us] (51.66%; 99.96%)
+ InferType: 24557us [24557us] (48.12%; 93.15%)
@@ -262,10 +262,10 @@ Refer to following sections and :py:func:`tvm.instrument.pass_instrument` for th
.. code-block:: none
Printing results of timing profile...
- InferType: 23684us [23684us] (48.39%; 48.39%)
- FoldScaleAxis: 25257us [7us] (51.61%; 51.61%)
- FoldConstant: 25250us [1620us] (51.59%; 99.97%)
- InferType: 23630us [23630us] (48.28%; 93.59%)
+ InferType: 24297us [24297us] (48.27%; 48.27%)
+ FoldScaleAxis: 26036us [9us] (51.73%; 51.73%)
+ FoldConstant: 26028us [1782us] (51.71%; 99.97%)
+ InferType: 24246us [24246us] (48.17%; 93.15%)
diff --git a/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt b/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt
index fd5c299228..ca93d276f4 100644
--- a/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt
@@ -331,7 +331,7 @@ latency of convolution.
.. code-block:: none
- Convolution: 40.173534 ms
+ Convolution: 40.106815 ms
diff --git a/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt b/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt
index ab50189aac..97f59b42f2 100644
--- a/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt
@@ -598,7 +598,7 @@ be able to run on our build server
.. code-block:: none
- conv2d with tensor core: 11.293082 ms
+ conv2d with tensor core: 12.222477 ms
diff --git a/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt b/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt
index a217c84a0b..b2ee4b769c 100644
--- a/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt
@@ -134,8 +134,8 @@ Then we write a baseline implementation, the simplest way to write a matrix mult
.. code-block:: none
- Numpy running time: 0.015800
- Baseline: 3.246315
+ Numpy running time: 0.014205
+ Baseline: 3.276621
@@ -227,7 +227,7 @@ fill 32 * 32 * sizeof(float) which is 4KB in the cache whose total size is 32KB
.. code-block:: none
- Opt1: 0.286214
+ Opt1: 0.285582
@@ -318,7 +318,7 @@ In this tutorial, we chose to vectorize the inner loop row data since it is cach
.. code-block:: none
- Opt2: 0.256444
+ Opt2: 0.264834
@@ -406,7 +406,7 @@ the access pattern for A matrix is more cache friendly.
.. code-block:: none
- Opt3: 0.110091
+ Opt3: 0.108351
@@ -523,7 +523,7 @@ flattening.
.. code-block:: none
- Opt4: 0.103922
+ Opt4: 0.103553
@@ -635,7 +635,7 @@ write to C when all the block results are ready.
.. code-block:: none
- Opt5: 0.097830
+ Opt5: 0.094406
@@ -748,7 +748,7 @@ Furthermore, we can also utilize multi-core processors to do the thread-level pa
.. code-block:: none
- Opt6: 0.116669
+ Opt6: 0.113181
diff --git a/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt b/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt
index c52ae9c722..723792ecb5 100644
--- a/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt
@@ -5,12 +5,12 @@
Computation times
=================
-**00:31.991** total execution time for **how_to_optimize_operators** files:
+**00:31.475** total execution time for **how_to_optimize_operators** files:
+-----------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_optimize_operators_opt_gemm.py` (``opt_gemm.py``) | 00:28.649 | 0.0 MB |
+| :ref:`sphx_glr_how_to_optimize_operators_opt_gemm.py` (``opt_gemm.py``) | 00:28.372 | 0.0 MB |
+-----------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_tensorcore.py` (``opt_conv_tensorcore.py``) | 00:02.046 | 0.0 MB |
+| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_tensorcore.py` (``opt_conv_tensorcore.py``) | 00:01.959 | 0.0 MB |
+-----------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_cuda.py` (``opt_conv_cuda.py``) | 00:01.296 | 0.0 MB |
+| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_cuda.py` (``opt_conv_cuda.py``) | 00:01.144 | 0.0 MB |
+-----------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt
index de6331260a..5ccb90611f 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt
@@ -5,18 +5,18 @@
Computation times
=================
-**03:36.431** total execution time for **how_to_tune_with_autoscheduler** files:
+**03:39.260** total execution time for **how_to_tune_with_autoscheduler** files:
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_x86.py` (``tune_network_x86.py``) | 01:34.448 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_x86.py` (``tune_network_x86.py``) | 01:35.480 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_cuda.py` (``tune_network_cuda.py``) | 01:10.986 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_cuda.py` (``tune_network_cuda.py``) | 01:12.473 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_arm.py` (``tune_network_arm.py``) | 00:19.548 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_arm.py` (``tune_network_arm.py``) | 00:20.013 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_mali.py` (``tune_network_mali.py``) | 00:16.609 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_mali.py` (``tune_network_mali.py``) | 00:16.920 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_conv2d_layer_cuda.py` (``tune_conv2d_layer_cuda.py``) | 00:14.741 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_conv2d_layer_cuda.py` (``tune_conv2d_layer_cuda.py``) | 00:14.272 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_sparse_x86.py` (``tune_sparse_x86.py``) | 00:00.098 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_sparse_x86.py` (``tune_sparse_x86.py``) | 00:00.101 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_arm.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_arm.rst.txt
index e12ab57ae9..2ce80c2a33 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_arm.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_arm.rst.txt
@@ -74,7 +74,7 @@ __name__ == "__main__":` block.
-.. GENERATED FROM PYTHON SOURCE LINES 62-74
+.. GENERATED FROM PYTHON SOURCE LINES 62-73
Define a Network
----------------
@@ -87,9 +87,8 @@ For convolutional neural networks, although auto-scheduler can work correctly
with any layout, we found the best performance is typically achieved with NHWC layout.
We also implemented more optimizations for NHWC layout with the auto-scheduler.
So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
-You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
-.. GENERATED FROM PYTHON SOURCE LINES 74-138
+.. GENERATED FROM PYTHON SOURCE LINES 73-137
.. code-block:: default
@@ -164,7 +163,7 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
-.. GENERATED FROM PYTHON SOURCE LINES 139-163
+.. GENERATED FROM PYTHON SOURCE LINES 138-162
Start RPC Tracker
-----------------
@@ -191,7 +190,7 @@ The expected output is
INFO:RPCTracker:bind to 0.0.0.0:9190
-.. GENERATED FROM PYTHON SOURCE LINES 165-207
+.. GENERATED FROM PYTHON SOURCE LINES 164-206
Register Devices to RPC Tracker
-----------------------------------
@@ -236,7 +235,7 @@ the output can be
You can register multiple devices to the tracker to accelerate the measurement in tuning.
-.. GENERATED FROM PYTHON SOURCE LINES 209-215
+.. GENERATED FROM PYTHON SOURCE LINES 208-214
Set Tuning Options
------------------
@@ -245,7 +244,7 @@ as example with a 64bit OS (Ubuntu 20.04). In your setting, you should modify th
and device_key accordingly.
set :code:`use_ndk` to True if you use android phone.
-.. GENERATED FROM PYTHON SOURCE LINES 215-242
+.. GENERATED FROM PYTHON SOURCE LINES 214-241
.. code-block:: default
@@ -283,7 +282,7 @@ set :code:`use_ndk` to True if you use android phone.
-.. GENERATED FROM PYTHON SOURCE LINES 243-252
+.. GENERATED FROM PYTHON SOURCE LINES 242-251
Extract Search Tasks
--------------------
@@ -295,7 +294,7 @@ as :code:`sum(latency[t] * weight[t])`, where :code:`latency[t]` is the
latency of a task and :code:`weight[t]` is the weight of the task.
The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 252-266
+.. GENERATED FROM PYTHON SOURCE LINES 251-265
.. code-block:: default
@@ -533,7 +532,7 @@ The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 267-286
+.. GENERATED FROM PYTHON SOURCE LINES 266-285
Tuning and Evaluation
---------------------
@@ -555,7 +554,7 @@ After auto-tuning, we can compile the network with the best schedules we found.
All measurement records are dumped into the log file during auto-tuning,
so we can read the log file and load the best schedules.
-.. GENERATED FROM PYTHON SOURCE LINES 286-351
+.. GENERATED FROM PYTHON SOURCE LINES 285-350
.. code-block:: default
@@ -631,7 +630,7 @@ so we can read the log file and load the best schedules.
-.. GENERATED FROM PYTHON SOURCE LINES 352-403
+.. GENERATED FROM PYTHON SOURCE LINES 351-402
.. note:: Explaining the printed information during tuning
@@ -685,7 +684,7 @@ so we can read the log file and load the best schedules.
errors are isolated from the main process.
-.. GENERATED FROM PYTHON SOURCE LINES 405-411
+.. GENERATED FROM PYTHON SOURCE LINES 404-410
.. note:: Terminate the tuning earlier
@@ -694,7 +693,7 @@ so we can read the log file and load the best schedules.
you should be able to do the compilation (the secion below).
-.. GENERATED FROM PYTHON SOURCE LINES 413-429
+.. GENERATED FROM PYTHON SOURCE LINES 412-428
Other Tips
----------
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt
index c080178c0f..61f4ee6dec 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt
@@ -70,7 +70,7 @@ __name__ == "__main__":` block.
-.. GENERATED FROM PYTHON SOURCE LINES 56-68
+.. GENERATED FROM PYTHON SOURCE LINES 56-67
Define a Network
----------------
@@ -83,9 +83,8 @@ For convolutional neural networks, although auto-scheduler can work correctly
with any layout, we found the best performance is typically achieved with NHWC layout.
We also implemented more optimizations for NHWC layout with the auto-scheduler.
So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
-You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
-.. GENERATED FROM PYTHON SOURCE LINES 68-128
+.. GENERATED FROM PYTHON SOURCE LINES 67-127
.. code-block:: default
@@ -156,7 +155,7 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
-.. GENERATED FROM PYTHON SOURCE LINES 129-138
+.. GENERATED FROM PYTHON SOURCE LINES 128-137
Extract Search Tasks
--------------------
@@ -168,7 +167,7 @@ as :code:`sum(latency[t] * weight[t])`, where :code:`latency[t]` is the
latency of a task and :code:`weight[t]` is the weight of the task.
The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 138-148
+.. GENERATED FROM PYTHON SOURCE LINES 137-147
.. code-block:: default
@@ -468,7 +467,7 @@ The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 149-171
+.. GENERATED FROM PYTHON SOURCE LINES 148-170
Begin Tuning
------------
@@ -493,7 +492,7 @@ Now, we set some options for tuning and launch the search tasks
:any:`auto_scheduler.LocalRPCMeasureContext` for more parameters.
-.. GENERATED FROM PYTHON SOURCE LINES 171-193
+.. GENERATED FROM PYTHON SOURCE LINES 170-192
.. code-block:: default
@@ -526,7 +525,7 @@ Now, we set some options for tuning and launch the search tasks
-.. GENERATED FROM PYTHON SOURCE LINES 194-247
+.. GENERATED FROM PYTHON SOURCE LINES 193-246
.. note:: Explain the printed information during tuning
@@ -582,7 +581,7 @@ Now, we set some options for tuning and launch the search tasks
errors are isolated from the main process.
-.. GENERATED FROM PYTHON SOURCE LINES 249-255
+.. GENERATED FROM PYTHON SOURCE LINES 248-254
.. note:: Terminate the tuning earlier
@@ -591,7 +590,7 @@ Now, we set some options for tuning and launch the search tasks
you should be able to do the compilation (the secion below).
-.. GENERATED FROM PYTHON SOURCE LINES 258-263
+.. GENERATED FROM PYTHON SOURCE LINES 257-262
Compile and Evaluate
--------------------
@@ -599,7 +598,7 @@ After auto-tuning, we can compile the network with the best schedules we found.
All measurement records are dumped into the log file during auto-tuning,
so we can read the log file and load the best schedules.
-.. GENERATED FROM PYTHON SOURCE LINES 263-281
+.. GENERATED FROM PYTHON SOURCE LINES 262-280
.. code-block:: default
@@ -633,12 +632,12 @@ so we can read the log file and load the best schedules.
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 3.2006 3.2006 3.2008 3.2003 0.0002
+ 3.2081 3.2079 3.2092 3.2070 0.0009
-.. GENERATED FROM PYTHON SOURCE LINES 282-298
+.. GENERATED FROM PYTHON SOURCE LINES 281-297
Other Tips
----------
@@ -660,7 +659,7 @@ Other Tips
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 10.986 seconds)
+ **Total running time of the script:** ( 1 minutes 12.473 seconds)
.. _sphx_glr_download_how_to_tune_with_autoscheduler_tune_network_cuda.py:
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_mali.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_mali.rst.txt
index 225dabb3fb..08e35619a7 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_mali.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_mali.rst.txt
@@ -73,7 +73,7 @@ __name__ == "__main__":` block.
-.. GENERATED FROM PYTHON SOURCE LINES 59-71
+.. GENERATED FROM PYTHON SOURCE LINES 59-70
Define a Network
----------------
@@ -86,9 +86,8 @@ For convolutional neural networks, although auto-scheduler can work correctly
with any layout, we found the best performance is typically achieved with NHWC layout.
We also implemented more optimizations for NHWC layout with the auto-scheduler.
So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
-You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
-.. GENERATED FROM PYTHON SOURCE LINES 71-137
+.. GENERATED FROM PYTHON SOURCE LINES 70-136
.. code-block:: default
@@ -165,7 +164,7 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
-.. GENERATED FROM PYTHON SOURCE LINES 138-143
+.. GENERATED FROM PYTHON SOURCE LINES 137-142
Start an RPC Tracker and Register Devices to the Tracker
--------------------------------------------------------
@@ -173,7 +172,7 @@ Please refer to the "Start RPC Tracker" and "Register Devices to RPC Tracker" se
in this :ref:`tutorial <tutorials-autotvm-start-rpc-tracker>` to start an RPC tracker
and register devices to the tracker.
-.. GENERATED FROM PYTHON SOURCE LINES 143-148
+.. GENERATED FROM PYTHON SOURCE LINES 142-147
.. code-block:: default
@@ -189,7 +188,7 @@ and register devices to the tracker.
-.. GENERATED FROM PYTHON SOURCE LINES 149-158
+.. GENERATED FROM PYTHON SOURCE LINES 148-157
Extract Search Tasks
--------------------
@@ -201,7 +200,7 @@ as :code:`sum(latency[t] * weight[t])`, where :code:`latency[t]` is the
latency of a task and :code:`weight[t]` is the weight of the task.
The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 158-167
+.. GENERATED FROM PYTHON SOURCE LINES 157-166
.. code-block:: default
@@ -419,7 +418,7 @@ The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 168-194
+.. GENERATED FROM PYTHON SOURCE LINES 167-193
.. note:: How to get the hardware parameters from remote device
@@ -448,7 +447,7 @@ The task scheduler will just optimize this objective.
)
-.. GENERATED FROM PYTHON SOURCE LINES 196-212
+.. GENERATED FROM PYTHON SOURCE LINES 195-211
Tuning and Evaluate
-------------------
@@ -467,7 +466,7 @@ Now, we set some options for tuning, launch the search tasks and evaluate the en
:any:`auto_scheduler.LocalRunner` for more parameters.
-.. GENERATED FROM PYTHON SOURCE LINES 212-265
+.. GENERATED FROM PYTHON SOURCE LINES 211-264
.. code-block:: default
@@ -531,7 +530,7 @@ Now, we set some options for tuning, launch the search tasks and evaluate the en
-.. GENERATED FROM PYTHON SOURCE LINES 266-324
+.. GENERATED FROM PYTHON SOURCE LINES 265-323
.. note:: Explain the printed information during tuning
@@ -592,7 +591,7 @@ Now, we set some options for tuning, launch the search tasks and evaluate the en
errors are isolated from the main process.
-.. GENERATED FROM PYTHON SOURCE LINES 326-332
+.. GENERATED FROM PYTHON SOURCE LINES 325-331
.. note:: Terminate the tuning earlier
@@ -601,7 +600,7 @@ Now, we set some options for tuning, launch the search tasks and evaluate the en
you should be able to do the compilation (the secion below).
-.. GENERATED FROM PYTHON SOURCE LINES 334-350
+.. GENERATED FROM PYTHON SOURCE LINES 333-349
Other Tips
----------
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt
index 92c89d3fe1..ab9dbf68f2 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt
@@ -72,7 +72,7 @@ __name__ == "__main__":` block.
-.. GENERATED FROM PYTHON SOURCE LINES 59-71
+.. GENERATED FROM PYTHON SOURCE LINES 59-70
Define a Network
----------------
@@ -85,9 +85,8 @@ For convolutional neural networks, although auto-scheduler can work correctly
with any layout, we found the best performance is typically achieved with NHWC layout.
We also implemented more optimizations for NHWC layout with the auto-scheduler.
So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
-You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
-.. GENERATED FROM PYTHON SOURCE LINES 71-146
+.. GENERATED FROM PYTHON SOURCE LINES 70-145
.. code-block:: default
@@ -173,7 +172,7 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
-.. GENERATED FROM PYTHON SOURCE LINES 147-156
+.. GENERATED FROM PYTHON SOURCE LINES 146-155
Extract Search Tasks
--------------------
@@ -185,7 +184,7 @@ as :code:`sum(latency[t] * weight[t])`, where :code:`latency[t]` is the
latency of a task and :code:`weight[t]` is the weight of the task.
The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 156-174
+.. GENERATED FROM PYTHON SOURCE LINES 155-173
.. code-block:: default
@@ -479,7 +478,7 @@ The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 175-191
+.. GENERATED FROM PYTHON SOURCE LINES 174-190
Begin Tuning
------------
@@ -498,7 +497,7 @@ Now, we set some options for tuning and launch the search tasks
:any:`auto_scheduler.LocalRunner` for more parameters.
-.. GENERATED FROM PYTHON SOURCE LINES 191-225
+.. GENERATED FROM PYTHON SOURCE LINES 190-224
.. code-block:: default
@@ -543,7 +542,7 @@ Now, we set some options for tuning and launch the search tasks
-.. GENERATED FROM PYTHON SOURCE LINES 226-284
+.. GENERATED FROM PYTHON SOURCE LINES 225-283
.. note:: Explain the printed information during tuning
@@ -604,7 +603,7 @@ Now, we set some options for tuning and launch the search tasks
errors are isolated from the main process.
-.. GENERATED FROM PYTHON SOURCE LINES 286-292
+.. GENERATED FROM PYTHON SOURCE LINES 285-291
.. note:: Terminate the tuning earlier
@@ -613,7 +612,7 @@ Now, we set some options for tuning and launch the search tasks
you should be able to do the compilation (the secion below).
-.. GENERATED FROM PYTHON SOURCE LINES 295-300
+.. GENERATED FROM PYTHON SOURCE LINES 294-299
Compile and Evaluate
--------------------
@@ -621,7 +620,7 @@ After auto-tuning, we can compile the network with the best schedules we found.
All measurement records are dumped into the log file during auto-tuning,
so we can read the log file and load the best schedules.
-.. GENERATED FROM PYTHON SOURCE LINES 300-318
+.. GENERATED FROM PYTHON SOURCE LINES 299-317
.. code-block:: default
@@ -655,12 +654,12 @@ so we can read the log file and load the best schedules.
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 705.2023 705.5045 706.6326 703.4697 1.3088
+ 709.6547 708.8154 711.6203 708.5286 1.3948
-.. GENERATED FROM PYTHON SOURCE LINES 319-335
+.. GENERATED FROM PYTHON SOURCE LINES 318-334
Other Tips
----------
@@ -682,7 +681,7 @@ Other Tips
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 34.448 seconds)
+ **Total running time of the script:** ( 1 minutes 35.480 seconds)
.. _sphx_glr_download_how_to_tune_with_autoscheduler_tune_network_x86.py:
diff --git a/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt b/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt
index 0f7124b9f1..41daee541a 100644
--- a/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt
@@ -5,12 +5,12 @@
Computation times
=================
-**00:22.695** total execution time for **how_to_tune_with_autotvm** files:
+**00:22.485** total execution time for **how_to_tune_with_autotvm** files:
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_conv2d_cuda.py` (``tune_conv2d_cuda.py``) | 00:22.658 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_conv2d_cuda.py` (``tune_conv2d_cuda.py``) | 00:22.453 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_relay_x86.py` (``tune_relay_x86.py``) | 00:00.020 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_relay_x86.py` (``tune_relay_x86.py``) | 00:00.016 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_relay_arm.py` (``tune_relay_arm.py``) | 00:00.006 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt b/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt
index 6642cd2c7c..03ca64babe 100644
--- a/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt
+++ b/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt
@@ -326,7 +326,7 @@ and measure running time.
Best config:
,None
- Time cost of this operator: 0.037185
+ Time cost of this operator: 0.037266
diff --git a/docs/_sources/how_to/tutorials/cross_compilation_and_rpc.rst.txt b/docs/_sources/how_to/tutorials/cross_compilation_and_rpc.rst.txt
index 1fcf0867ca..06e9f5da75 100644
--- a/docs/_sources/how_to/tutorials/cross_compilation_and_rpc.rst.txt
+++ b/docs/_sources/how_to/tutorials/cross_compilation_and_rpc.rst.txt
@@ -274,7 +274,7 @@ device and returns the measured cost. Network overhead is excluded.
.. code-block:: none
- 1.206e-07 secs/op
+ 1.199e-07 secs/op
diff --git a/docs/_sources/how_to/tutorials/optimize_llm.rst.txt b/docs/_sources/how_to/tutorials/optimize_llm.rst.txt
index bf69584156..acb79454d8 100644
--- a/docs/_sources/how_to/tutorials/optimize_llm.rst.txt
+++ b/docs/_sources/how_to/tutorials/optimize_llm.rst.txt
@@ -858,7 +858,7 @@ IRModule to generate the token.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 8.145 seconds)
+ **Total running time of the script:** ( 1 minutes 8.876 seconds)
.. _sphx_glr_download_how_to_tutorials_optimize_llm.py:
diff --git a/docs/_sources/how_to/tutorials/sg_execution_times.rst.txt b/docs/_sources/how_to/tutorials/sg_execution_times.rst.txt
index 7b030022ae..51db927a6f 100644
--- a/docs/_sources/how_to/tutorials/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/tutorials/sg_execution_times.rst.txt
@@ -5,14 +5,14 @@
Computation times
=================
-**01:10.587** total execution time for **how_to_tutorials** files:
+**01:11.297** total execution time for **how_to_tutorials** files:
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tutorials_optimize_llm.py` (``optimize_llm.py``) | 01:08.145 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tutorials_optimize_llm.py` (``optimize_llm.py``) | 01:08.876 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tutorials_cross_compilation_and_rpc.py` (``cross_compilation_and_rpc.py``) | 00:00.888 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tutorials_customize_opt.py` (``customize_opt.py``) | 00:00.825 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tutorials_customize_opt.py` (``customize_opt.py``) | 00:00.850 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tutorials_cross_compilation_and_rpc.py` (``cross_compilation_and_rpc.py``) | 00:00.799 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tutorials_e2e_opt_model.py` (``e2e_opt_model.py``) | 00:00.705 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tutorials_e2e_opt_model.py` (``e2e_opt_model.py``) | 00:00.796 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt b/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt
index 30c2a37d1a..1f839b8408 100644
--- a/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt
@@ -360,10 +360,10 @@ Timing the untuned program
########## Build without Autotuning ##########
Node Name Ops Time(us) Time(%) Shape Inputs Outputs Measurements(us)
--------- --- -------- ------- ----- ------ ------- ----------------
- tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 299.1 98.748 (1, 2, 10, 10, 3) 2 1 [299.1]
- tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 2.86 0.944 (1, 6, 10, 10) 1 1 [2.86]
- tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 0.933 0.308 (1, 1, 10, 10, 3) 1 1 [0.933]
- Total_time - 302.894 - - - - -
+ tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 292.9 98.738 (1, 2, 10, 10, 3) 2 1 [292.9]
+ tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 2.802 0.945 (1, 6, 10, 10) 1 1 [2.802]
+ tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 0.941 0.317 (1, 1, 10, 10, 3) 1 1 [0.941]
+ Total_time - 296.644 - - - - -
@@ -428,10 +428,10 @@ Timing the tuned program
########## Build with Autotuning ##########
Node Name Ops Time(us) Time(%) Shape Inputs Outputs Measurements(us)
--------- --- -------- ------- ----- ------ ------- ----------------
- tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 132.8 98.102 (1, 6, 10, 10, 1) 2 1 [132.8]
- tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 1.725 1.274 (1, 6, 10, 10) 1 1 [1.725]
- tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 0.844 0.624 (1, 3, 10, 10, 1) 1 1 [0.844]
- Total_time - 135.369 - - - - -
+ tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 131.4 98.101 (1, 6, 10, 10, 1) 2 1 [131.4]
+ tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 1.721 1.285 (1, 6, 10, 10) 1 1 [1.721]
+ tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 0.823 0.614 (1, 3, 10, 10, 1) 1 1 [0.823]
+ Total_time - 133.944 - - - - -
@@ -439,7 +439,7 @@ Timing the tuned program
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 23.605 seconds)
+ **Total running time of the script:** ( 1 minutes 24.692 seconds)
.. _sphx_glr_download_how_to_work_with_microtvm_micro_autotune.py:
diff --git a/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt b/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt
index c4e5ff23a8..26e6a644d3 100644
--- a/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt
@@ -118,7 +118,7 @@ download a cat image and preprocess it to use as the model input.
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torch/ao/quantization/utils.py:376: UserWarning: must run observer before calling calculate_qparams. Returning default values.
warnings.warn(
Downloading: "https://download.pytorch.org/models/quantized/mobilenet_v2_qnnpack_37f702c5.pth" to /workspace/.cache/torch/hub/checkpoints/mobilenet_v2_qnnpack_37f702c5.pth
- 0%| | 0.00/3.42M [00:00<?, ?B/s] 100%|##########| 3.42M/3.42M [00:00<00:00, 68.9MB/s]
+ 0%| | 0.00/3.42M [00:00<?, ?B/s] 100%|##########| 3.42M/3.42M [00:00<00:00, 93.1MB/s]
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torch/ao/nn/quantized/modules/__init__.py:98: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
return torch.quantize_per_tensor(X, float(self.scale),
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torch/ao/nn/quantized/modules/__init__.py:99: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
@@ -324,7 +324,7 @@ Look up prediction top 1 index in 1000 class synset.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 27.633 seconds)
+ **Total running time of the script:** ( 1 minutes 29.005 seconds)
.. _sphx_glr_download_how_to_work_with_microtvm_micro_pytorch.py:
diff --git a/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt b/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt
index 81bcf55574..6e17614104 100644
--- a/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt
@@ -218,7 +218,7 @@ take about **2 minutes** to download the Stanford Cars, while COCO 2017 validati
.. code-block:: none
- '/tmp/tmpv58525sy/images/random'
+ '/tmp/tmp45hld4xx/images/random'
@@ -318,8 +318,8 @@ objects to other stuff? We can display some examples from our datasets using ``m
.. code-block:: none
- /tmp/tmpv58525sy/images/target contains 8144 images
- /tmp/tmpv58525sy/images/random contains 5000 images
+ /tmp/tmp45hld4xx/images/target contains 8144 images
+ /tmp/tmp45hld4xx/images/random contains 5000 images
@@ -494,13 +494,13 @@ the time on our validation set).
.. code-block:: none
Epoch 1/3
- 328/328 - 43s - loss: 0.2262 - accuracy: 0.9207 - val_loss: 0.1104 - val_accuracy: 0.9645 - 43s/epoch - 132ms/step
+ 328/328 - 43s - loss: 0.2078 - accuracy: 0.9267 - val_loss: 0.1254 - val_accuracy: 0.9566 - 43s/epoch - 132ms/step
Epoch 2/3
- 328/328 - 40s - loss: 0.1018 - accuracy: 0.9629 - val_loss: 0.1065 - val_accuracy: 0.9634 - 40s/epoch - 121ms/step
+ 328/328 - 41s - loss: 0.0962 - accuracy: 0.9647 - val_loss: 0.1096 - val_accuracy: 0.9664 - 41s/epoch - 124ms/step
Epoch 3/3
- 328/328 - 40s - loss: 0.0664 - accuracy: 0.9744 - val_loss: 0.1095 - val_accuracy: 0.9675 - 40s/epoch - 121ms/step
+ 328/328 - 40s - loss: 0.0652 - accuracy: 0.9751 - val_loss: 0.1010 - val_accuracy: 0.9690 - 40s/epoch - 122ms/step
- <keras.callbacks.History object at 0x7faa0f37a5b0>
+ <keras.callbacks.History object at 0x7f409d0739d0>
@@ -861,7 +861,7 @@ Arduino tutorial for how to do that `on GitHub <https://github.com/guberti/tvm-a
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 5 minutes 10.365 seconds)
+ **Total running time of the script:** ( 5 minutes 4.291 seconds)
.. _sphx_glr_download_how_to_work_with_microtvm_micro_train.py:
diff --git a/docs/_sources/how_to/work_with_microtvm/micro_tvmc.rst.txt b/docs/_sources/how_to/work_with_microtvm/micro_tvmc.rst.txt
index 5af1b65144..d8b692f7ab 100644
--- a/docs/_sources/how_to/work_with_microtvm/micro_tvmc.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/micro_tvmc.rst.txt
@@ -95,7 +95,7 @@ Compiling a TFLite model to a Model Library Format
Model Library Format (MLF) is an output format that TVM provides for micro targets. MLF is a tarball
containing a file for each piece of the TVM compiler output which can be used on micro targets outside
- TVM environment. Read more about :ref:`Model Library Format <model_library_format>`.
+ TVM environment.
Here, we generate a MLF file for ``qemu_x86`` Zephyr board. You can chooses `aot` or `graph` executor type
to run this tutorial, however, we recommend to use `aot` for microTVM targets since `aot` uses ahead of time
diff --git a/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt b/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt
index 1369ea33c3..569ab36554 100644
--- a/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt
@@ -5,20 +5,20 @@
Computation times
=================
-**08:31.240** total execution time for **how_to_work_with_microtvm** files:
+**08:28.379** total execution time for **how_to_work_with_microtvm** files:
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_train.py` (``micro_train.py``) | 05:10.365 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_train.py` (``micro_train.py``) | 05:04.291 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_pytorch.py` (``micro_pytorch.py``) | 01:27.633 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_pytorch.py` (``micro_pytorch.py``) | 01:29.005 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_autotune.py` (``micro_autotune.py``) | 01:23.605 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_autotune.py` (``micro_autotune.py``) | 01:24.692 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_aot.py` (``micro_aot.py``) | 00:11.600 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_aot.py` (``micro_aot.py``) | 00:11.871 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_custom_ide.py` (``micro_custom_ide.py``) | 00:10.092 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_custom_ide.py` (``micro_custom_ide.py``) | 00:10.261 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_tflite.py` (``micro_tflite.py``) | 00:07.944 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_tflite.py` (``micro_tflite.py``) | 00:08.259 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_work_with_microtvm_micro_ethosu.py` (``micro_ethosu.py``) | 00:00.000 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt b/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt
index b4b9cd6e40..8159c6001f 100644
--- a/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt
@@ -5,14 +5,14 @@
Computation times
=================
-**00:39.129** total execution time for **how_to_work_with_relay** files:
+**00:40.128** total execution time for **how_to_work_with_relay** files:
+----------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_relay_using_pipeline_executor.py` (``using_pipeline_executor.py``) | 00:33.685 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_relay_using_pipeline_executor.py` (``using_pipeline_executor.py``) | 00:34.200 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_relay_using_external_lib.py` (``using_external_lib.py``) | 00:03.659 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_relay_using_external_lib.py` (``using_external_lib.py``) | 00:03.777 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_relay_build_gcn.py` (``build_gcn.py``) | 00:01.779 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_relay_build_gcn.py` (``build_gcn.py``) | 00:02.145 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_work_with_relay_using_relay_viz.py` (``using_relay_viz.py``) | 00:00.006 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt b/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt
index a47d7aed0e..4171cf8f47 100644
--- a/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt
@@ -5,16 +5,16 @@
Computation times
=================
-**00:03.075** total execution time for **how_to_work_with_schedules** files:
+**00:03.109** total execution time for **how_to_work_with_schedules** files:
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_tensorize.py` (``tensorize.py``) | 00:01.407 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_tensorize.py` (``tensorize.py``) | 00:01.460 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_reduction.py` (``reduction.py``) | 00:00.695 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_reduction.py` (``reduction.py``) | 00:00.684 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_scan.py` (``scan.py``) | 00:00.686 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_scan.py` (``scan.py``) | 00:00.676 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_extern_op.py` (``extern_op.py``) | 00:00.124 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_extern_op.py` (``extern_op.py``) | 00:00.126 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_work_with_schedules_schedule_primitives.py` (``schedule_primitives.py``) | 00:00.068 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/index.rst.txt b/docs/_sources/index.rst.txt
index 2102bdd33a..3abc39e82f 100644
--- a/docs/_sources/index.rst.txt
+++ b/docs/_sources/index.rst.txt
@@ -54,6 +54,7 @@ driving its costs down.
:maxdepth: 2
:caption: Deep Dive
+ arch/index
deep_dive/tensor_ir/index
deep_dive/relax/index
@@ -73,7 +74,6 @@ driving its costs down.
dev/tutorial/index
dev/how_to/how_to.rst
reference/langref/index
- arch/index
topic/microtvm/index
topic/vta/index
diff --git a/docs/_sources/reference/langref/relay_expr.rst.txt b/docs/_sources/reference/langref/relay_expr.rst.txt
index c50acc2949..c789331efe 100644
--- a/docs/_sources/reference/langref/relay_expr.rst.txt
+++ b/docs/_sources/reference/langref/relay_expr.rst.txt
@@ -540,9 +540,7 @@ the graph node will only be evaluated once by the compiled program.
These bindings allow for a style of programming that corresponds to that already
employed by NNVM and other dataflow graph-based input formats. The fact that the variables
are not scoped offers some flexibility in evaluation order compared to :code:`let`
-bindings, though this can also introduce some ambiguity in programs (the
-:ref:`developer introduction to the Relay IR<relay-dev-intro>` includes more detailed discussion
-of this nuance).
+bindings, though this can also introduce some ambiguity in programs.
*Note: Graph bindings are not currently parsed by the text format.*
diff --git a/docs/_sources/topic/microtvm/index.rst.txt b/docs/_sources/topic/microtvm/index.rst.txt
index 4dd4ab5d51..2bac70241d 100644
--- a/docs/_sources/topic/microtvm/index.rst.txt
+++ b/docs/_sources/topic/microtvm/index.rst.txt
@@ -58,13 +58,6 @@ more as they follow through them. Here is a list of tutorials that you can start
3. Try running a more complex tutorial: :ref:`Creating Your MLPerfTiny Submission with microTVM <tutorial-micro-mlperftiny>`.
-How microTVM Works
-~~~~~~~~~~~~~~~~~~
-
-
-You can read more about the design of these pieces at the :ref:`microTVM Design Document <microTVM-design>`.
-
-
Help and Discussion
~~~~~~~~~~~~~~~~~~~
diff --git a/docs/_sources/topic/vta/tutorials/frontend/deploy_detection.rst.txt b/docs/_sources/topic/vta/tutorials/frontend/deploy_detection.rst.txt
index 3953c247b4..215c81a113 100644
--- a/docs/_sources/topic/vta/tutorials/frontend/deploy_detection.rst.txt
+++ b/docs/_sources/topic/vta/tutorials/frontend/deploy_detection.rst.txt
@@ -337,7 +337,7 @@ The compilation steps are:
/workspace/python/tvm/relay/build_module.py:345: DeprecationWarning: Please use input parameter mod (tvm.IRModule) instead of deprecated parameter mod (tvm.relay.function.Function)
warnings.warn(
- yolov3-tiny inference graph built in 27.14s!
+ yolov3-tiny inference graph built in 27.58s!
@@ -447,7 +447,7 @@ Download test image
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 3.222 seconds)
+ **Total running time of the script:** ( 1 minutes 3.481 seconds)
.. _sphx_glr_download_topic_vta_tutorials_frontend_deploy_detection.py:
diff --git a/docs/_sources/topic/vta/tutorials/frontend/sg_execution_times.rst.txt b/docs/_sources/topic/vta/tutorials/frontend/sg_execution_times.rst.txt
index 70b896f0d6..d57776f19e 100644
--- a/docs/_sources/topic/vta/tutorials/frontend/sg_execution_times.rst.txt
+++ b/docs/_sources/topic/vta/tutorials/frontend/sg_execution_times.rst.txt
@@ -5,8 +5,8 @@
Computation times
=================
-**01:03.222** total execution time for **topic_vta_tutorials_frontend** files:
+**01:03.481** total execution time for **topic_vta_tutorials_frontend** files:
+--------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_frontend_deploy_detection.py` (``deploy_detection.py``) | 01:03.222 | 0.0 MB |
+| :ref:`sphx_glr_topic_vta_tutorials_frontend_deploy_detection.py` (``deploy_detection.py``) | 01:03.481 | 0.0 MB |
+--------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/topic/vta/tutorials/optimize/sg_execution_times.rst.txt b/docs/_sources/topic/vta/tutorials/optimize/sg_execution_times.rst.txt
index 78d3a71509..45dae019b9 100644
--- a/docs/_sources/topic/vta/tutorials/optimize/sg_execution_times.rst.txt
+++ b/docs/_sources/topic/vta/tutorials/optimize/sg_execution_times.rst.txt
@@ -5,10 +5,10 @@
Computation times
=================
-**00:03.217** total execution time for **topic_vta_tutorials_optimize** files:
+**00:03.210** total execution time for **topic_vta_tutorials_optimize** files:
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_optimize_convolution_opt.py` (``convolution_opt.py``) | 00:02.709 | 0.0 MB |
+| :ref:`sphx_glr_topic_vta_tutorials_optimize_convolution_opt.py` (``convolution_opt.py``) | 00:02.719 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_optimize_matrix_multiply_opt.py` (``matrix_multiply_opt.py``) | 00:00.508 | 0.0 MB |
+| :ref:`sphx_glr_topic_vta_tutorials_optimize_matrix_multiply_opt.py` (``matrix_multiply_opt.py``) | 00:00.491 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/topic/vta/tutorials/sg_execution_times.rst.txt b/docs/_sources/topic/vta/tutorials/sg_execution_times.rst.txt
index 8341d80e67..8b386ef1af 100644
--- a/docs/_sources/topic/vta/tutorials/sg_execution_times.rst.txt
+++ b/docs/_sources/topic/vta/tutorials/sg_execution_times.rst.txt
@@ -5,10 +5,10 @@
Computation times
=================
-**00:00.861** total execution time for **topic_vta_tutorials** files:
+**00:00.851** total execution time for **topic_vta_tutorials** files:
+---------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_matrix_multiply.py` (``matrix_multiply.py``) | 00:00.466 | 0.0 MB |
+| :ref:`sphx_glr_topic_vta_tutorials_matrix_multiply.py` (``matrix_multiply.py``) | 00:00.475 | 0.0 MB |
+---------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_vta_get_started.py` (``vta_get_started.py``) | 00:00.394 | 0.0 MB |
+| :ref:`sphx_glr_topic_vta_tutorials_vta_get_started.py` (``vta_get_started.py``) | 00:00.375 | 0.0 MB |
+---------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt b/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt
index c6c99f6cb3..c21c4c270c 100644
--- a/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt
+++ b/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt
@@ -207,13 +207,6 @@ trials, we can load the best schedule from the log file and apply it.
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- *E
-
-
@@ -325,7 +318,7 @@ We build the binary and check its correctness and performance.
.. code-block:: none
- Execution time of this operator: 90.760 ms
+ Execution time of this operator: 90.674 ms
@@ -441,7 +434,7 @@ operations.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 54.163 seconds)
+ **Total running time of the script:** ( 1 minutes 27.501 seconds)
.. _sphx_glr_download_tutorial_auto_scheduler_matmul_x86.py:
diff --git a/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt b/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt
index e078e8d589..0dee3c1cb0 100644
--- a/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt
+++ b/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt
@@ -454,16 +454,16 @@ reduce variance, we take 5 measurements and average them.
waiting for device...
device available
Get devices for measurement successfully!
- No: 1 GFLOPS: 16.36/16.36 result: MeasureResult(costs=(0.0164080454,), error_no=MeasureErrorNo.NO_ERROR, all_cost=1.6350281238555908, timestamp=1727115705.8567064) [('tile_y', [-1, 16]), ('tile_x', [-1, 64])],None,64
- No: 2 GFLOPS: 2.01/16.36 result: MeasureResult(costs=(0.13345639380000002,), error_no=MeasureErrorNo.NO_ERROR, all_cost=3.485071897506714, timestamp=1727115708.2577577) [('tile_y', [-1, 512]), ('tile_x', [-1, 4])],None,29
- No: 3 GFLOPS: 3.02/16.36 result: MeasureResult(costs=(0.08899330280000001,), error_no=MeasureErrorNo.NO_ERROR, all_cost=2.757917642593384, timestamp=1727115709.946948) [('tile_y', [-1, 16]), ('tile_x', [-1, 2])],None,14
- No: 4 GFLOPS: 0.53/16.36 result: MeasureResult(costs=(0.5103244109999999,), error_no=MeasureErrorNo.NO_ERROR, all_cost=9.517277479171753, timestamp=1727115718.3903072) [('tile_y', [-1, 128]), ('tile_x', [-1, 1])],None,7
- No: 5 GFLOPS: 11.14/16.36 result: MeasureResult(costs=(0.0240911296,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.7468953132629395, timestamp=1727115719.250925) [('tile_y', [-1, 512]), ('tile_x', [-1, 64])],None,69
- No: 6 GFLOPS: 10.81/16.36 result: MeasureResult(costs=(0.0248216032,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6860897541046143, timestamp=1727115719.9093401) [('tile_y', [-1, 256]), ('tile_x', [-1, 32])],None,58
- No: 7 GFLOPS: 10.94/16.36 result: MeasureResult(costs=(0.0245421142,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6861560344696045, timestamp=1727115720.5664396) [('tile_y', [-1, 4]), ('tile_x', [-1, 16])],None,42
- No: 8 GFLOPS: 12.73/16.36 result: MeasureResult(costs=(0.0210909356,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6308143138885498, timestamp=1727115721.174006) [('tile_y', [-1, 16]), ('tile_x', [-1, 256])],None,84
- No: 9 GFLOPS: 1.39/16.36 result: MeasureResult(costs=(0.1934425936,), error_no=MeasureErrorNo.NO_ERROR, all_cost=3.317202568054199, timestamp=1727115724.6252258) [('tile_y', [-1, 2]), ('tile_x', [-1, 1])],None,1
- No: 10 GFLOPS: 13.09/16.36 result: MeasureResult(costs=(0.020507102399999998,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.5743050575256348, timestamp=1727115725.2142715) [('tile_y', [-1, 256]), ('tile_x', [-1, 64])],None,68
+ No: 1 GFLOPS: 1.39/1.39 result: MeasureResult(costs=(0.19271322640000002,), error_no=MeasureErrorNo.NO_ERROR, all_cost=4.443490743637085, timestamp=1727142829.1783926) [('tile_y', [-1, 2]), ('tile_x', [-1, 1])],None,1
+ No: 2 GFLOPS: 6.34/6.34 result: MeasureResult(costs=(0.0423128034,), error_no=MeasureErrorNo.NO_ERROR, all_cost=2.045419692993164, timestamp=1727142830.1141634) [('tile_y', [-1, 16]), ('tile_x', [-1, 4])],None,24
+ No: 3 GFLOPS: 8.60/8.60 result: MeasureResult(costs=(0.031214836200000002,), error_no=MeasureErrorNo.NO_ERROR, all_cost=1.8656024932861328, timestamp=1727142830.877827) [('tile_y', [-1, 2]), ('tile_x', [-1, 8])],None,31
+ No: 4 GFLOPS: 3.76/8.60 result: MeasureResult(costs=(0.0713352662,), error_no=MeasureErrorNo.NO_ERROR, all_cost=2.494560480117798, timestamp=1727142832.2789276) [('tile_y', [-1, 256]), ('tile_x', [-1, 8])],None,38
+ No: 5 GFLOPS: 10.92/10.92 result: MeasureResult(costs=(0.024571276,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6777307987213135, timestamp=1727142833.1059942) [('tile_y', [-1, 4]), ('tile_x', [-1, 16])],None,42
+ No: 6 GFLOPS: 4.37/10.92 result: MeasureResult(costs=(0.06141663039999999,), error_no=MeasureErrorNo.NO_ERROR, all_cost=1.26688551902771, timestamp=1727142834.348876) [('tile_y', [-1, 8]), ('tile_x', [-1, 2])],None,13
+ No: 7 GFLOPS: 11.66/11.66 result: MeasureResult(costs=(0.023029409999999997,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.7064568996429443, timestamp=1727142834.990033) [('tile_y', [-1, 1]), ('tile_x', [-1, 128])],None,70
+ No: 8 GFLOPS: 9.97/11.66 result: MeasureResult(costs=(0.026928206399999998,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.7137658596038818, timestamp=1727142835.677348) [('tile_y', [-1, 4]), ('tile_x', [-1, 32])],None,52
+ No: 9 GFLOPS: 1.05/11.66 result: MeasureResult(costs=(0.25602315400000003,), error_no=MeasureErrorNo.NO_ERROR, all_cost=4.324889659881592, timestamp=1727142840.126668) [('tile_y', [-1, 64]), ('tile_x', [-1, 2])],None,16
+ No: 10 GFLOPS: 11.36/11.66 result: MeasureResult(costs=(0.0236358544,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6257641315460205, timestamp=1727142840.770762) [('tile_y', [-1, 256]), ('tile_x', [-1, 256])],None,88
diff --git a/docs/_sources/tutorial/autotvm_relay_x86.rst.txt b/docs/_sources/tutorial/autotvm_relay_x86.rst.txt
index 8746f0f5e9..7e163b2f49 100644
--- a/docs/_sources/tutorial/autotvm_relay_x86.rst.txt
+++ b/docs/_sources/tutorial/autotvm_relay_x86.rst.txt
@@ -311,7 +311,7 @@ standard deviation.
.. code-block:: none
- {'mean': 465.9675887499907, 'median': 464.822291100063, 'std': 1.9676390636516226}
+ {'mean': 467.68376613997134, 'median': 467.5010677499813, 'std': 1.4039108912338887}
@@ -582,30 +582,30 @@ the tuning data to.
.. code-block:: none
- [Task 1/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 1/25] Current/Best: 15.39/ 23.57 GFLOPS | Progress: (4/20) | 8.91 s [Task 1/25] Current/Best: 12.87/ 23.68 GFLOPS | Progress: (8/20) | 14.65 s [Task 1/25] Current/Best: 20.64/ 23.68 GFLOPS | Progress: (12/20) | 16.84 s [Task 1/25] Current/Best: 8.85/ 23.68 GFLOPS | Progress: (16/20) | 18.97 s [Task 1/25] Current/Best: 14.74/ 23.68 GFLOPS | Progress: (20/20) | 26.70 s Done.
- [Task 2/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 2/25] Current/Best: 5.27/ 18.60 GFLOPS | Progress: (4/20) | 4.05 s [Task 2/25] Current/Best: 17.55/ 18.60 GFLOPS | Progress: (8/20) | 6.79 s [Task 2/25] Current/Best: 13.28/ 21.20 GFLOPS | Progress: (12/20) | 8.66 s [Task 2/25] Current/Best: 18.71/ 21.20 GFLOPS | Progress: (16/20) | 10.37 s [Task 2/25] Current/Best: 22.52/ 22.52 GFLOPS | Progress: (20/20) | 11.89 s Done.
- [Task 3/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 3/25] Current/Best: 10.52/ 15.71 GFLOPS | Progress: (4/20) | 5.26 s [Task 3/25] Current/Best: 19.38/ 19.75 GFLOPS | Progress: (8/20) | 7.63 s [Task 3/25] Current/Best: 15.37/ 19.75 GFLOPS | Progress: (12/20) | 10.47 s [Task 3/25] Current/Best: 18.07/ 19.91 GFLOPS | Progress: (16/20) | 12.83 s [Task 3/25] Current/Best: 12.28/ 19.91 GFLOPS | Progress: (20/20) | 16.19 s Done.
- [Task 4/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 4/25] Current/Best: 16.55/ 16.55 GFLOPS | Progress: (4/20) | 6.84 s [Task 4/25] Current/Best: 10.83/ 21.76 GFLOPS | Progress: (8/20) | 11.91 s [Task 4/25] Current/Best: 15.64/ 21.76 GFLOPS | Progress: (12/20) | 13.85 s [Task 4/25] Current/Best: 7.48/ 21.76 GFLOPS | Progress: (16/20) | 16.01 s [Task 4/25] Current/Best: 21.18/ 21.76 GFLOPS | Progress: (20/20) | 17.90 s Done.
- [Task 5/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 5/25] Current/Best: 21.48/ 21.48 GFLOPS | Progress: (4/20) | 4.23 s [Task 5/25] Current/Best: 23.54/ 23.54 GFLOPS | Progress: (8/20) | 6.09 s [Task 5/25] Current/Best: 16.09/ 23.54 GFLOPS | Progress: (12/20) | 8.17 s [Task 5/25] Current/Best: 14.70/ 23.54 GFLOPS | Progress: (16/20) | 10.15 s [Task 5/25] Current/Best: 17.30/ 23.54 GFLOPS | Progress: (20/20) | 12.20 s Done.
- [Task 6/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 6/25] Current/Best: 20.28/ 20.28 GFLOPS | Progress: (4/20) | 5.23 s [Task 6/25] Current/Best: 3.16/ 20.28 GFLOPS | Progress: (8/20) | 8.05 s [Task 6/25] Current/Best: 19.06/ 20.28 GFLOPS | Progress: (12/20) | 10.95 s [Task 6/25] Current/Best: 17.54/ 20.28 GFLOPS | Progress: (16/20) | 13.57 s [Task 6/25] Current/Best: 6.96/ 21.48 GFLOPS | Progress: (20/20) | 15.96 s Done.
- [Task 7/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 7/25] Current/Best: 5.88/ 16.91 GFLOPS | Progress: (4/20) | 5.58 s [Task 7/25] Current/Best: 14.46/ 22.89 GFLOPS | Progress: (8/20) | 7.88 s [Task 7/25] Current/Best: 10.38/ 22.89 GFLOPS | Progress: (12/20) | 10.42 s [Task 7/25] Current/Best: 10.69/ 22.89 GFLOPS | Progress: (16/20) | 12.92 s [Task 7/25] Current/Best: 15.51/ 22.89 GFLOPS | Progress: (20/20) | 14.91 s Done.
- [Task 8/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 8/25] Current/Best: 5.91/ 13.02 GFLOPS | Progress: (4/20) | 14.00 s [Task 8/25] Current/Best: 11.59/ 13.02 GFLOPS | Progress: (8/20) | 24.94 s [Task 8/25] Current/Best: 10.16/ 19.58 GFLOPS | Progress: (12/20) | 31.07 s [Task 8/25] Current/Best: 8.93/ 19.58 GFLOPS | Progress: (16/20) | 42.93 s [Task 8/25] Current/Best: 1.75/ 19.58 GFLOPS | Progress: (20/20) | 55.72 s [Task 9/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 9/25] Current/Best: 8.87/ 21.17 GFLOPS | Progress: (4/20) | 10.14 s [Task 9/25] Current/Best: 19.64/ 21.17 GFLOPS | Progress: (8/20) | 12.75 s [Task 9/25] Current/Best: 18.28/ 21.17 GFLOPS | Progress: (12/20) | 15.99 s [Task 9/25] Current/Best: 13.21/ 21.31 GFLOPS | Progress: (16/20) | 20.58 s [Task 9/25] Current/Best: 21.83/ 21.83 GFLOPS | Progress: (2
0/20) | 25.38 s Done.
- [Task 10/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 10/25] Current/Best: 9.77/ 21.17 GFLOPS | Progress: (4/20) | 5.47 s [Task 10/25] Current/Best: 5.49/ 21.17 GFLOPS | Progress: (8/20) | 7.62 s [Task 10/25] Current/Best: 11.34/ 21.17 GFLOPS | Progress: (12/20) | 10.51 s [Task 10/25] Current/Best: 5.21/ 21.17 GFLOPS | Progress: (16/20) | 12.68 s [Task 10/25] Current/Best: 5.45/ 21.17 GFLOPS | Progress: (20/20) | 14.51 s Done.
- [Task 11/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 11/25] Current/Best: 10.95/ 24.98 GFLOPS | Progress: (4/20) | 4.62 s [Task 11/25] Current/Best: 25.01/ 25.01 GFLOPS | Progress: (8/20) | 6.94 s [Task 11/25] Current/Best: 23.76/ 25.01 GFLOPS | Progress: (12/20) | 9.71 s [Task 11/25] Current/Best: 7.45/ 25.01 GFLOPS | Progress: (16/20) | 11.91 s [Task 11/25] Current/Best: 6.39/ 25.01 GFLOPS | Progress: (20/20) | 14.30 s Done.
- [Task 12/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 12/25] Current/Best: 15.07/ 15.07 GFLOPS | Progress: (4/20) | 5.51 s [Task 12/25] Current/Best: 5.25/ 20.43 GFLOPS | Progress: (8/20) | 7.81 s [Task 12/25] Current/Best: 16.13/ 20.43 GFLOPS | Progress: (12/20) | 11.09 s [Task 12/25] Current/Best: 11.23/ 20.43 GFLOPS | Progress: (16/20) | 16.18 s [Task 12/25] Current/Best: 9.98/ 20.43 GFLOPS | Progress: (20/20) | 21.22 s Done.
- [Task 13/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 13/25] Current/Best: 19.39/ 19.83 GFLOPS | Progress: (4/20) | 4.99 s [Task 13/25] Current/Best: 12.60/ 19.83 GFLOPS | Progress: (8/20) | 8.41 s [Task 13/25] Current/Best: 14.11/ 19.83 GFLOPS | Progress: (12/20) | 11.53 s [Task 13/25] Current/Best: 22.96/ 22.96 GFLOPS | Progress: (16/20) | 14.76 s [Task 13/25] Current/Best: 16.97/ 22.96 GFLOPS | Progress: (20/20) | 18.75 s Done.
- [Task 14/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 14/25] Current/Best: 6.88/ 14.21 GFLOPS | Progress: (4/20) | 11.21 s [Task 14/25] Current/Best: 8.55/ 19.85 GFLOPS | Progress: (8/20) | 13.49 s [Task 14/25] Current/Best: 14.66/ 22.32 GFLOPS | Progress: (12/20) | 20.21 s [Task 14/25] Current/Best: 9.83/ 22.32 GFLOPS | Progress: (16/20) | 23.76 s [Task 14/25] Current/Best: 19.70/ 22.32 GFLOPS | Progress: (20/20) | 32.10 s Done.
- [Task 15/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 15/25] Current/Best: 16.41/ 22.78 GFLOPS | Progress: (4/20) | 10.21 s [Task 15/25] Current/Best: 7.18/ 22.87 GFLOPS | Progress: (8/20) | 17.82 s [Task 15/25] Current/Best: 14.12/ 22.87 GFLOPS | Progress: (12/20) | 28.90 s [Task 15/25] Current/Best: 13.85/ 22.87 GFLOPS | Progress: (16/20) | 33.06 s [Task 15/25] Current/Best: 19.06/ 22.87 GFLOPS | Progress: (20/20) | 35.37 s [Task 16/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 16/25] Current/Best: 6.13/ 18.16 GFLOPS | Progress: (4/20) | 4.16 s [Task 16/25] Current/Best: 22.07/ 22.07 GFLOPS | Progress: (8/20) | 6.73 s [Task 16/25] Current/Best: 18.90/ 22.07 GFLOPS | Progress: (12/20) | 8.33 s [Task 16/25] Current/Best: 16.40/ 22.07 GFLOPS | Progress: (16/20) | 11.45 s Done.
- Done.
- [Task 16/25] Current/Best: 6.05/ 22.07 GFLOPS | Progress: (20/20) | 13.23 s Done.
- [Task 17/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 17/25] Current/Best: 5.44/ 22.98 GFLOPS | Progress: (4/20) | 5.55 s [Task 17/25] Current/Best: 14.45/ 23.88 GFLOPS | Progress: (8/20) | 7.56 s [Task 17/25] Current/Best: 6.40/ 23.88 GFLOPS | Progress: (12/20) | 10.21 s [Task 17/25] Current/Best: 17.43/ 23.88 GFLOPS | Progress: (16/20) | 13.65 s [Task 17/25] Current/Best: 20.08/ 23.88 GFLOPS | Progress: (20/20) | 15.95 s Done.
- [Task 18/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 18/25] Current/Best: 9.75/ 11.59 GFLOPS | Progress: (4/20) | 8.51 s [Task 18/25] Current/Best: 7.99/ 12.21 GFLOPS | Progress: (8/20) | 16.10 s [Task 18/25] Current/Best: 4.25/ 18.82 GFLOPS | Progress: (12/20) | 18.41 s [Task 18/25] Current/Best: 13.17/ 18.82 GFLOPS | Progress: (16/20) | 21.09 s [Task 18/25] Current/Best: 15.20/ 18.82 GFLOPS | Progress: (20/20) | 27.12 s Done.
- [Task 19/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 19/25] Current/Best: 6.43/ 17.59 GFLOPS | Progress: (4/20) | 8.76 s [Task 19/25] Current/Best: 10.61/ 20.60 GFLOPS | Progress: (8/20) | 11.11 s [Task 19/25] Current/Best: 18.63/ 20.60 GFLOPS | Progress: (12/20) | 13.93 s [Task 19/25] Current/Best: 7.59/ 20.88 GFLOPS | Progress: (16/20) | 18.70 s [Task 19/25] Current/Best: 18.82/ 21.37 GFLOPS | Progress: (20/20) | 21.72 s Done.
- [Task 20/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 20/25] Current/Best: 19.20/ 19.20 GFLOPS | Progress: (4/20) | 4.87 s [Task 20/25] Current/Best: 1.61/ 19.20 GFLOPS | Progress: (8/20) | 15.47 s [Task 20/25] Current/Best: 18.35/ 19.20 GFLOPS | Progress: (12/20) | 18.54 s [Task 20/25] Current/Best: 11.68/ 19.20 GFLOPS | Progress: (16/20) | 21.35 s [Task 20/25] Current/Best: 10.65/ 19.20 GFLOPS | Progress: (20/20) | 32.89 s [Task 21/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 21/25] Current/Best: 11.37/ 19.20 GFLOPS | Progress: (4/20) | 7.06 s [Task 21/25] Current/Best: 12.20/ 19.20 GFLOPS | Progress: (8/20) | 17.97 s [Task 21/25] Current/Best: 14.79/ 19.20 GFLOPS | Progress: (12/20) | 30.61 s [Task 21/25] Current/Best: 20.22/ 20.22 GFLOPS | Progress: (16/20) | 36.54 s [Task 21/25] Current/Best: 7.63/ 20.22 GFLOPS | Progress: (20/
20) | 44.15 s [Task 22/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 22/25] Current/Best: 17.64/ 21.46 GFLOPS | Progress: (4/20) | 4.87 s [Task 22/25] Current/Best: 16.93/ 21.46 GFLOPS | Progress: (8/20) | 7.72 s [Task 22/25] Current/Best: 18.73/ 21.46 GFLOPS | Progress: (12/20) | 11.75 s [Task 22/25] Current/Best: 13.15/ 21.93 GFLOPS | Progress: (16/20) | 13.78 s [Task 22/25] Current/Best: 7.79/ 22.13 GFLOPS | Progress: (20/20) | 16.36 s Done.
- [Task 23/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 23/25] Current/Best: 5.53/ 12.30 GFLOPS | Progress: (4/20) | 5.99 s [Task 23/25] Current/Best: 18.33/ 19.72 GFLOPS | Progress: (8/20) | 8.41 s [Task 23/25] Current/Best: 11.86/ 19.72 GFLOPS | Progress: (12/20) | 13.12 s [Task 23/25] Current/Best: 5.29/ 22.80 GFLOPS | Progress: (16/20) | 15.66 s [Task 23/25] Current/Best: 19.16/ 22.80 GFLOPS | Progress: (20/20) | 19.31 s Done.
- [Task 24/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 24/25] Current/Best: 3.87/ 7.73 GFLOPS | Progress: (4/20) | 13.42 s [Task 24/25] Current/Best: 4.17/ 7.73 GFLOPS | Progress: (8/20) | 24.48 s Done.
- Done.
- [Task 24/25] Current/Best: 3.41/ 7.94 GFLOPS | Progress: (12/20) | 35.55 s [Task 24/25] Current/Best: 3.02/ 7.94 GFLOPS | Progress: (16/20) | 46.62 s [Task 24/25] Current/Best: 2.31/ 7.94 GFLOPS | Progress: (20/20) | 51.42 s [Task 25/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 25/25] Current/Best: 2.24/ 3.78 GFLOPS | Progress: (4/20) | 13.65 s [Task 25/25] Current/Best: 2.70/ 9.33 GFLOPS | Progress: (8/20) | 20.61 s [Task 25/25] Current/Best: 4.63/ 9.33 GFLOPS | Progress: (12/20) | 22.32 s [Task 25/25] Current/Best: 8.53/ 9.34 GFLOPS | Progress: (16/20) | 33.30 s [Task 25/25] Current/Best: 6.52/ 9.34 GFLOPS | Progress: (20/20) | 37.36 s
+ [Task 1/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 1/25] Current/Best: 6.82/ 19.42 GFLOPS | Progress: (4/20) | 9.66 s [Task 1/25] Current/Best: 13.48/ 19.42 GFLOPS | Progress: (8/20) | 12.74 s [Task 1/25] Current/Best: 18.65/ 19.91 GFLOPS | Progress: (12/20) | 14.77 s [Task 1/25] Current/Best: 7.65/ 24.06 GFLOPS | Progress: (16/20) | 17.53 s [Task 1/25] Current/Best: 17.99/ 24.06 GFLOPS | Progress: (20/20) | 20.96 s Done.
+ [Task 2/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 2/25] Current/Best: 14.34/ 14.34 GFLOPS | Progress: (4/20) | 4.96 s [Task 2/25] Current/Best: 20.36/ 20.36 GFLOPS | Progress: (8/20) | 6.37 s [Task 2/25] Current/Best: 18.51/ 20.36 GFLOPS | Progress: (12/20) | 7.79 s [Task 2/25] Current/Best: 19.38/ 20.36 GFLOPS | Progress: (16/20) | 9.57 s [Task 2/25] Current/Best: 18.40/ 20.36 GFLOPS | Progress: (20/20) | 10.98 s Done.
+ [Task 3/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 3/25] Current/Best: 20.64/ 23.01 GFLOPS | Progress: (4/20) | 4.88 s [Task 3/25] Current/Best: 6.92/ 23.01 GFLOPS | Progress: (8/20) | 7.78 s [Task 3/25] Current/Best: 7.68/ 23.01 GFLOPS | Progress: (12/20) | 9.99 s [Task 3/25] Current/Best: 12.53/ 24.62 GFLOPS | Progress: (16/20) | 12.52 s [Task 3/25] Current/Best: 7.24/ 24.62 GFLOPS | Progress: (20/20) | 14.91 s Done.
+ [Task 4/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 4/25] Current/Best: 6.84/ 11.08 GFLOPS | Progress: (4/20) | 5.57 s [Task 4/25] Current/Best: 13.10/ 16.02 GFLOPS | Progress: (8/20) | 8.96 s [Task 4/25] Current/Best: 14.56/ 17.24 GFLOPS | Progress: (12/20) | 10.70 s [Task 4/25] Current/Best: 11.45/ 22.60 GFLOPS | Progress: (16/20) | 13.43 s [Task 4/25] Current/Best: 11.01/ 22.60 GFLOPS | Progress: (20/20) | 15.52 s Done.
+ [Task 5/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 5/25] Current/Best: 18.24/ 18.24 GFLOPS | Progress: (4/20) | 4.46 s [Task 5/25] Current/Best: 21.70/ 21.70 GFLOPS | Progress: (8/20) | 6.78 s [Task 5/25] Current/Best: 17.63/ 21.70 GFLOPS | Progress: (12/20) | 8.85 s [Task 5/25] Current/Best: 11.93/ 21.70 GFLOPS | Progress: (16/20) | 10.87 s [Task 5/25] Current/Best: 11.99/ 21.70 GFLOPS | Progress: (20/20) | 13.01 s Done.
+ [Task 6/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 6/25] Current/Best: 11.85/ 18.22 GFLOPS | Progress: (4/20) | 5.68 s [Task 6/25] Current/Best: 21.24/ 21.24 GFLOPS | Progress: (8/20) | 8.28 s [Task 6/25] Current/Best: 15.80/ 21.90 GFLOPS | Progress: (12/20) | 11.32 s [Task 6/25] Current/Best: 13.71/ 21.90 GFLOPS | Progress: (16/20) | 13.58 s [Task 6/25] Current/Best: 13.65/ 21.90 GFLOPS | Progress: (20/20) | 15.75 s Done.
+ [Task 7/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 7/25] Current/Best: 22.77/ 22.77 GFLOPS | Progress: (4/20) | 4.69 s [Task 7/25] Current/Best: 16.17/ 22.77 GFLOPS | Progress: (8/20) | 7.18 s [Task 7/25] Current/Best: 19.15/ 22.77 GFLOPS | Progress: (12/20) | 10.08 s [Task 7/25] Current/Best: 13.55/ 22.77 GFLOPS | Progress: (16/20) | 12.47 s [Task 7/25] Current/Best: 9.90/ 22.77 GFLOPS | Progress: (20/20) | 14.77 s Done.
+ [Task 8/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 8/25] Current/Best: 13.81/ 13.81 GFLOPS | Progress: (4/20) | 5.47 s [Task 8/25] Current/Best: 5.91/ 13.81 GFLOPS | Progress: (8/20) | 9.91 s [Task 8/25] Current/Best: 9.06/ 14.01 GFLOPS | Progress: (12/20) | 19.04 s [Task 8/25] Current/Best: 16.73/ 16.73 GFLOPS | Progress: (16/20) | 21.51 s [Task 8/25] Current/Best: 8.81/ 16.73 GFLOPS | Progress: (20/20) | 31.03 s Done.
+ [Task 9/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 9/25] Current/Best: 15.71/ 21.73 GFLOPS | Progress: (4/20) | 4.52 s [Task 9/25] Current/Best: 18.45/ 21.73 GFLOPS | Progress: (8/20) | 12.21 s [Task 9/25] Current/Best: 9.05/ 21.73 GFLOPS | Progress: (12/20) | 16.84 s [Task 9/25] Current/Best: 11.22/ 21.73 GFLOPS | Progress: (16/20) | 19.50 s [Task 9/25] Current/Best: 8.93/ 21.73 GFLOPS | Progress: (20/20) | 24.69 s Done.
+ [Task 10/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 10/25] Current/Best: 5.12/ 16.46 GFLOPS | Progress: (4/20) | 4.58 s [Task 10/25] Current/Best: 4.49/ 16.46 GFLOPS | Progress: (8/20) | 6.64 s [Task 10/25] Current/Best: 21.25/ 21.25 GFLOPS | Progress: (12/20) | 8.35 s [Task 10/25] Current/Best: 13.31/ 21.25 GFLOPS | Progress: (16/20) | 10.65 s [Task 10/25] Current/Best: 6.39/ 21.25 GFLOPS | Progress: (20/20) | 13.10 s Done.
+ [Task 11/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 11/25] Current/Best: 19.45/ 23.81 GFLOPS | Progress: (4/20) | 4.84 s [Task 11/25] Current/Best: 13.38/ 23.81 GFLOPS | Progress: (8/20) | 8.10 s [Task 11/25] Current/Best: 9.15/ 23.81 GFLOPS | Progress: (12/20) | 10.68 s [Task 11/25] Current/Best: 15.84/ 23.81 GFLOPS | Progress: (16/20) | 12.62 s [Task 11/25] Current/Best: 22.73/ 23.81 GFLOPS | Progress: (20/20) | 15.27 s Done.
+ [Task 12/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 12/25] Current/Best: 17.13/ 17.13 GFLOPS | Progress: (4/20) | 5.76 s [Task 12/25] Current/Best: 11.34/ 18.82 GFLOPS | Progress: (8/20) | 9.98 s [Task 12/25] Current/Best: 16.40/ 18.82 GFLOPS | Progress: (12/20) | 13.21 s [Task 12/25] Current/Best: 9.26/ 18.82 GFLOPS | Progress: (16/20) | 15.74 s [Task 12/25] Current/Best: 22.09/ 22.09 GFLOPS | Progress: (20/20) | 18.25 s Done.
+ [Task 13/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 13/25] Current/Best: 10.87/ 16.75 GFLOPS | Progress: (4/20) | 6.43 s [Task 13/25] Current/Best: 23.18/ 23.18 GFLOPS | Progress: (8/20) | 8.66 s [Task 13/25] Current/Best: 12.75/ 23.18 GFLOPS | Progress: (12/20) | 13.06 s [Task 13/25] Current/Best: 7.77/ 23.18 GFLOPS | Progress: (16/20) | 16.63 s [Task 13/25] Current/Best: 13.40/ 23.18 GFLOPS | Progress: (20/20) | 19.26 s Done.
+ [Task 14/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 14/25] Current/Best: 8.07/ 23.29 GFLOPS | Progress: (4/20) | 7.41 s [Task 14/25] Current/Best: 14.10/ 23.29 GFLOPS | Progress: (8/20) | 15.61 s [Task 14/25] Current/Best: 5.75/ 23.29 GFLOPS | Progress: (12/20) | 20.22 s [Task 14/25] Current/Best: 9.02/ 23.29 GFLOPS | Progress: (16/20) | 25.48 s [Task 14/25] Current/Best: 11.38/ 23.29 GFLOPS | Progress: (20/20) | 31.07 s Done.
+ [Task 15/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 15/25] Current/Best: 9.61/ 17.31 GFLOPS | Progress: (4/20) | 13.03 s [Task 15/25] Current/Best: 12.60/ 17.31 GFLOPS | Progress: (8/20) | 24.30 s [Task 15/25] Current/Best: 11.85/ 19.71 GFLOPS | Progress: (12/20) | 27.71 s [Task 15/25] Current/Best: 17.58/ 19.71 GFLOPS | Progress: (16/20) | 29.95 s [Task 15/25] Current/Best: 18.55/ 19.71 GFLOPS | Progress: (20/20) | 31.67 s [Task 16/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 16/25] Current/Best: 6.44/ 21.33 GFLOPS | Progress: (4/20) | 4.04 s [Task 16/25] Current/Best: 13.83/ 21.33 GFLOPS | Progress: (8/20) | 5.99 s [Task 16/25] Current/Best: 13.89/ 21.33 GFLOPS | Progress: (12/20) | 7.90 s [Task 16/25] Current/Best: 20.32/ 21.38 GFLOPS | Progress: (16/20) | 10.61 s [Task 16/25] Current/Best: 17.64/ 21.38 GFLOPS | Progress: (20/2
0) | 12.47 s Done.
+ [Task 17/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 17/25] Current/Best: 23.92/ 23.92 GFLOPS | Progress: (4/20) | 4.54 s [Task 17/25] Current/Best: 8.99/ 23.92 GFLOPS | Progress: (8/20) | 7.66 s [Task 17/25] Current/Best: 15.19/ 23.92 GFLOPS | Progress: (12/20) | 9.72 s [Task 17/25] Current/Best: 17.72/ 23.92 GFLOPS | Progress: (16/20) | 13.42 s [Task 17/25] Current/Best: 19.92/ 24.07 GFLOPS | Progress: (20/20) | 15.98 s Done.
+ [Task 18/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 18/25] Current/Best: 1.63/ 16.55 GFLOPS | Progress: (4/20) | 9.71 s [Task 18/25] Current/Best: 9.77/ 18.70 GFLOPS | Progress: (8/20) | 12.33 s [Task 18/25] Current/Best: 9.70/ 18.70 GFLOPS | Progress: (12/20) | 15.66 s [Task 18/25] Current/Best: 14.70/ 18.70 GFLOPS | Progress: (16/20) | 17.73 s [Task 18/25] Current/Best: 8.60/ 18.70 GFLOPS | Progress: (20/20) | 20.56 s Done.
+ [Task 19/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 19/25] Current/Best: 19.98/ 19.98 GFLOPS | Progress: (4/20) | 4.80 s [Task 19/25] Current/Best: 20.24/ 20.24 GFLOPS | Progress: (8/20) | 9.74 s [Task 19/25] Current/Best: 18.60/ 20.35 GFLOPS | Progress: (12/20) | 13.32 s [Task 19/25] Current/Best: 16.52/ 20.98 GFLOPS | Progress: (16/20) | 16.46 s [Task 19/25] Current/Best: 9.37/ 20.98 GFLOPS | Progress: (20/20) | 20.21 s Done.
+ [Task 20/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 20/25] Current/Best: 3.19/ 17.93 GFLOPS | Progress: (4/20) | 5.02 s [Task 20/25] Current/Best: 1.62/ 17.93 GFLOPS | Progress: (8/20) | 10.21 s [Task 20/25] Current/Best: 18.46/ 20.39 GFLOPS | Progress: (12/20) | 12.77 s [Task 20/25] Current/Best: 2.68/ 20.39 GFLOPS | Progress: (16/20) | 15.90 s [Task 20/25] Current/Best: 9.72/ 20.39 GFLOPS | Progress: (20/20) | 19.20 s Done.
+ [Task 21/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 21/25] Current/Best: 5.50/ 19.17 GFLOPS | Progress: (4/20) | 5.13 s [Task 21/25] Current/Best: 9.01/ 19.78 GFLOPS | Progress: (8/20) | 12.04 s [Task 21/25] Current/Best: 7.73/ 20.44 GFLOPS | Progress: (12/20) | 22.95 s [Task 21/25] Current/Best: 14.26/ 21.27 GFLOPS | Progress: (16/20) | 24.84 s [Task 21/25] Current/Best: 2.79/ 21.27 GFLOPS | Progress: (20/20) | 35.43 s Done.
+ [Task 22/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 22/25] Current/Best: 9.22/ 19.33 GFLOPS | Progress: (4/20) | 5.88 s [Task 22/25] Current/Best: 19.12/ 19.68 GFLOPS | Progress: (8/20) | 8.62 s [Task 22/25] Current/Best: 22.73/ 22.73 GFLOPS | Progress: (12/20) | 14.85 s [Task 22/25] Current/Best: 19.75/ 22.73 GFLOPS | Progress: (16/20) | 16.45 s [Task 22/25] Current/Best: 5.27/ 22.73 GFLOPS | Progress: (20/20) | 18.28 s Done.
+ [Task 23/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 23/25] Current/Best: 13.54/ 21.99 GFLOPS | Progress: (4/20) | 6.73 s [Task 23/25] Current/Best: 11.43/ 21.99 GFLOPS | Progress: (8/20) | 10.55 s [Task 23/25] Current/Best: 11.23/ 23.09 GFLOPS | Progress: (12/20) | 14.24 s [Task 23/25] Current/Best: 18.50/ 23.09 GFLOPS | Progress: (16/20) | 17.08 s [Task 23/25] Current/Best: 19.97/ 23.09 GFLOPS | Progress: (20/20) | 19.93 s Done.
+ [Task 24/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 24/25] Current/Best: 4.69/ 4.69 GFLOPS | Progress: (4/20) | 13.53 s [Task 24/25] Current/Best: 1.89/ 7.11 GFLOPS | Progress: (8/20) | 22.84 s [Task 24/25] Current/Best: 2.62/ 7.11 GFLOPS | Progress: (12/20) | 31.84 s Done.
+ [Task 24/25] Current/Best: 8.10/ 8.10 GFLOPS | Progress: (16/20) | 42.82 s [Task 24/25] Current/Best: 1.48/ 8.10 GFLOPS | Progress: (20/20) | 55.39 s [Task 25/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 25/25] Current/Best: 1.60/ 5.94 GFLOPS | Progress: (4/20) | 13.52 s [Task 25/25] Current/Best: 10.11/ 10.11 GFLOPS | Progress: (8/20) | 18.44 s [Task 25/25] Current/Best: 1.60/ 10.11 GFLOPS | Progress: (12/20) | 29.43 s [Task 25/25] Current/Best: 1.52/ 10.11 GFLOPS | Progress: (16/20) | 33.87 s [Task 25/25] Current/Best: 7.05/ 10.11 GFLOPS | Progress: (20/20) | 36.76 s
@@ -674,7 +674,6 @@ model using optimized operators to speed up our computations.
.. code-block:: none
Done.
- Done.
@@ -766,8 +765,8 @@ improvement in comparing the optimized model to the unoptimized model.
.. code-block:: none
- optimized: {'mean': 389.17297738000343, 'median': 389.22363794990815, 'std': 2.047359566546186}
- unoptimized: {'mean': 465.9675887499907, 'median': 464.822291100063, 'std': 1.9676390636516226}
+ optimized: {'mean': 380.05313241003023, 'median': 380.0628646000405, 'std': 0.9680927434579931}
+ unoptimized: {'mean': 467.68376613997134, 'median': 467.5010677499813, 'std': 1.4039108912338887}
@@ -790,7 +789,7 @@ profiling/benchmarking.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 14 minutes 30.325 seconds)
+ **Total running time of the script:** ( 13 minutes 15.999 seconds)
.. _sphx_glr_download_tutorial_autotvm_relay_x86.py:
diff --git a/docs/_sources/tutorial/intro_topi.rst.txt b/docs/_sources/tutorial/intro_topi.rst.txt
index f9fb0a7257..c5212b15c6 100644
--- a/docs/_sources/tutorial/intro_topi.rst.txt
+++ b/docs/_sources/tutorial/intro_topi.rst.txt
@@ -270,7 +270,7 @@ As you can see, scheduled stages of computation have been accumulated and we can
.. code-block:: none
- [stage(a, placeholder(a, 0x223d53c0)), stage(b, placeholder(b, 0x141bc150)), stage(T_add, compute(T_add, body=[a[ax0, ax1, ax2] + b[ax1, ax2]], axis=[T.iter_var(ax0, T.Range(0, 100), "DataPar", ""), T.iter_var(ax1, T.Range(0, 10), "DataPar", ""), T.iter_var(ax2, T.Range(0, 10), "DataPar", "")], reduce_axis=[], tag=broadcast, attrs={})), stage(T_multiply, compute(T_multiply, body=[a[ax0, ax1, ax2] * b[ax1, ax2]], axis=[T.iter_var(ax0, T.Range(0, 100), "DataPar", ""), T.iter_var(ax1, T [...]
+ [stage(a, placeholder(a, 0x27e24ae0)), stage(b, placeholder(b, 0x146296c0)), stage(T_add, compute(T_add, body=[a[ax0, ax1, ax2] + b[ax1, ax2]], axis=[T.iter_var(ax0, T.Range(0, 100), "DataPar", ""), T.iter_var(ax1, T.Range(0, 10), "DataPar", ""), T.iter_var(ax2, T.Range(0, 10), "DataPar", "")], reduce_axis=[], tag=broadcast, attrs={})), stage(T_multiply, compute(T_multiply, body=[a[ax0, ax1, ax2] * b[ax1, ax2]], axis=[T.iter_var(ax0, T.Range(0, 100), "DataPar", ""), T.iter_var(ax1, T [...]
diff --git a/docs/_sources/tutorial/sg_execution_times.rst.txt b/docs/_sources/tutorial/sg_execution_times.rst.txt
index 6f0263e54c..90f7084615 100644
--- a/docs/_sources/tutorial/sg_execution_times.rst.txt
+++ b/docs/_sources/tutorial/sg_execution_times.rst.txt
@@ -5,28 +5,28 @@
Computation times
=================
-**18:35.246** total execution time for **tutorial** files:
+**16:47.604** total execution time for **tutorial** files:
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_autotvm_relay_x86.py` (``autotvm_relay_x86.py``) | 14:30.325 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_autotvm_relay_x86.py` (``autotvm_relay_x86.py``) | 13:15.999 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_auto_scheduler_matmul_x86.py` (``auto_scheduler_matmul_x86.py``) | 01:54.163 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_auto_scheduler_matmul_x86.py` (``auto_scheduler_matmul_x86.py``) | 01:27.501 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_tensor_expr_get_started.py` (``tensor_expr_get_started.py``) | 00:59.158 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_tensor_expr_get_started.py` (``tensor_expr_get_started.py``) | 00:57.268 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_relay_quick_start.py` (``relay_quick_start.py``) | 00:42.899 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_relay_quick_start.py` (``relay_quick_start.py``) | 00:43.343 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_autotvm_matmul_x86.py` (``autotvm_matmul_x86.py``) | 00:27.108 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_autotvm_matmul_x86.py` (``autotvm_matmul_x86.py``) | 00:21.889 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_intro_topi.py` (``intro_topi.py``) | 00:00.859 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_intro_topi.py` (``intro_topi.py``) | 00:00.866 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_tensor_ir_blitz_course.py` (``tensor_ir_blitz_course.py``) | 00:00.734 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_tensor_ir_blitz_course.py` (``tensor_ir_blitz_course.py``) | 00:00.738 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_tutorial_uma.py` (``uma.py``) | 00:00.000 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_tutorial_introduction.py` (``introduction.py``) | 00:00.000 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_tvmc_command_line_driver.py` (``tvmc_command_line_driver.py``) | 00:00.000 | 0.0 MB |
-+------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_tutorial_tvmc_python.py` (``tvmc_python.py``) | 00:00.000 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
+| :ref:`sphx_glr_tutorial_tvmc_command_line_driver.py` (``tvmc_command_line_driver.py``) | 00:00.000 | 0.0 MB |
++------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/tutorial/tensor_expr_get_started.rst.txt b/docs/_sources/tutorial/tensor_expr_get_started.rst.txt
index 6377375e13..0c578ab974 100644
--- a/docs/_sources/tutorial/tensor_expr_get_started.rst.txt
+++ b/docs/_sources/tutorial/tensor_expr_get_started.rst.txt
@@ -285,7 +285,7 @@ helper function to run a profile of the TVM generated code.
.. code-block:: none
- Numpy running time: 0.000007
+ Numpy running time: 0.000006
naive: 0.000006
@@ -389,7 +389,7 @@ compile and run this new schedule with the parallel operation applied:
.. code-block:: none
- parallel: 0.000006
+ parallel: 0.000007
@@ -444,7 +444,7 @@ factor to be the number of threads on your CPU.
.. code-block:: none
- vector: 0.000039
+ vector: 0.000040
# from tvm.script import ir as I
# from tvm.script import tir as T
@@ -498,10 +498,10 @@ We can now compare the different schedules
.. code-block:: none
Operator Timing Performance
- numpy 6.504839984700084e-06 1.0
- naive 5.8745e-06 0.9030967731438906
- parallel 6.3345e-06 0.9738133474304153
- vector 3.86341e-05 5.939285223137013
+ numpy 6.12344001638121e-06 1.0
+ naive 5.6084e-06 0.9158904120880759
+ parallel 6.8707e-06 1.1220327106364636
+ vector 3.9827100000000004e-05 6.5040401952915285
@@ -922,7 +922,7 @@ matrix multiplication.
.. code-block:: none
- Numpy running time: 0.017456
+ Numpy running time: 0.014134
@@ -980,7 +980,7 @@ optimizations.
.. code-block:: none
- none: 3.300062
+ none: 3.288692
@@ -1080,7 +1080,7 @@ schedule.
.. code-block:: none
- blocking: 0.320758
+ blocking: 0.286527
@@ -1164,7 +1164,7 @@ already cache friendly from our previous optimizations.
.. code-block:: none
- vectorization: 0.315352
+ vectorization: 0.259926
# from tvm.script import ir as I
# from tvm.script import tir as T
@@ -1230,7 +1230,7 @@ more cache friendly.
.. code-block:: none
- loop permutation: 0.123749
+ loop permutation: 0.109613
# from tvm.script import ir as I
# from tvm.script import tir as T
@@ -1321,7 +1321,7 @@ optimized schedule.
.. code-block:: none
- array packing: 0.107664
+ array packing: 0.103193
# from tvm.script import ir as I
# from tvm.script import tir as T
@@ -1404,7 +1404,7 @@ to `C` when all the block results are ready.
.. code-block:: none
- block caching: 0.098843
+ block caching: 0.094209
# from tvm.script import ir as I
# from tvm.script import tir as T
@@ -1478,7 +1478,7 @@ of thread-level parallelization.
.. code-block:: none
- parallelization: 0.117472
+ parallelization: 0.113185
# from tvm.script import ir as I
# from tvm.script import tir as T
@@ -1548,13 +1548,13 @@ working, we can compare the results.
.. code-block:: none
Operator Timing Performance
- none 3.3000619232999995 1.0
- blocking 0.32075802789999996 0.0971975785167231
- vectorization 0.31535238190000003 0.09555953470856499
- loop permutation 0.12374914559999999 0.037499037435107636
- array packing 0.1076638932 0.03262480998912231
- block caching 0.0988430389 0.02995187399427912
- parallelization 0.1174721443 0.035596951520997545
+ none 3.2886916501 1.0
+ blocking 0.286526524 0.08712477619824513
+ vectorization 0.25992597529999995 0.07903628644907962
+ loop permutation 0.10961282480000001 0.03333022261198218
+ array packing 0.10319332340000001 0.031378230122870346
+ block caching 0.0942092035 0.028646408214383783
+ parallelization 0.11318534090000001 0.034416525762322035
diff --git a/docs/arch/benchmark.html b/docs/arch/benchmark.html
deleted file mode 100644
index 07ec2a4f86..0000000000
--- a/docs/arch/benchmark.html
+++ /dev/null
@@ -1,666 +0,0 @@
-
-
-
-
-
-
-<!DOCTYPE html>
-<html class="writer-html5" lang="en" >
-<head>
- <meta charset="utf-8">
-
- <meta name="viewport" content="width=device-width, initial-scale=1.0">
-
- <title>Benchmark Performance Log Format — tvm 0.18.dev0 documentation</title>
-
-
-
- <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
- <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
- <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
- <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-binder.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-dataframe.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-rendered-html.css" type="text/css" />
- <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
- <link rel="stylesheet" href="../_static/css/tlcpack_theme.css" type="text/css" />
-
-
-
- <link rel="shortcut icon" href="../_static/tvm-logo-square.png"/>
-
-
-
-
-
-
-
- <script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
- <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
- <script src="../_static/jquery.js"></script>
- <script src="../_static/underscore.js"></script>
- <script src="../_static/doctools.js"></script>
-
- <script type="text/javascript" src="../_static/js/theme.js"></script>
-
-
- <script type="text/javascript" src="../_static/js/tlcpack_theme.js"></script>
- <link rel="index" title="Index" href="../genindex.html" />
- <link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="TensorFlow Frontend" href="frontend/tensorflow.html" />
- <link rel="prev" title="Convert Layout Pass" href="convert_layout.html" />
-</head>
-
-<body class="wy-body-for-nav">
-
-
- <div class="wy-grid-for-nav">
-
-
-<header class="header">
- <div class="innercontainer">
- <div class="headerInner d-flex justify-content-between align-items-center">
- <div class="headerLogo">
- <a href="https://tvm.apache.org/"><img src=https://tvm.apache.org/assets/images/logo.svg alt="logo"></a>
- </div>
-
- <div id="headMenu" class="headerNav">
- <button type="button" id="closeHeadMenu" class="navCloseBtn"><img src="../_static/img/close-icon.svg" alt="Close"></button>
- <ul class="nav">
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/community>Community</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/download>Download</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/blog>Blog</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/docs>Docs</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvmconf.org>Conference</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://github.com/apache/tvm/>Github</a>
- </li>
- </ul>
- <div class="responsivetlcdropdown">
- <button type="button" class="btn-link">
- ASF
- </button>
- <ul>
- <li>
- <a href=https://apache.org/>Apache Homepage</a>
- </li>
- <li>
- <a href=https://www.apache.org/licenses/>License</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/sponsorship.html>Sponsorship</a>
- </li>
- <li>
- <a href=https://tvm.apache.org/docs/reference/security.html>Security</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/thanks.html>Thanks</a>
- </li>
- <li>
- <a href=https://www.apache.org/events/current-event>Events</a>
- </li>
- </ul>
- </div>
- </div>
- <div class="responsiveMenuIcon">
- <button type="button" id="menuBtn" class="btn-menu"><img src="../_static/img/menu-icon.svg" alt="Menu Icon"></button>
- </div>
-
- <div class="tlcDropdown">
- <div class="dropdown">
- <button type="button" class="btn-link dropdown-toggle" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
- ASF
- </button>
- <div class="dropdown-menu dropdown-menu-right">
- <ul>
- <li>
- <a href=https://apache.org/>Apache Homepage</a>
- </li>
- <li>
- <a href=https://www.apache.org/licenses/>License</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/sponsorship.html>Sponsorship</a>
- </li>
- <li>
- <a href=https://tvm.apache.org/docs/reference/security.html>Security</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/thanks.html>Thanks</a>
- </li>
- <li>
- <a href=https://www.apache.org/events/current-event>Events</a>
- </li>
- </ul>
- </div>
- </div>
- </div>
- </div>
- </div>
- </header>
-
- <nav data-toggle="wy-nav-shift" class="wy-nav-side fixed">
- <div class="wy-side-scroll">
- <div class="wy-side-nav-search" >
-
-
-
- <a href="../index.html">
-
-
-
-
- <img src="../_static/tvm-logo-small.png" class="logo" alt="Logo"/>
-
- </a>
-
-
-
-
- <input type="checkbox" class="version-toggle-box" hidden id="version-toggle">
- <label for="version-toggle" class="version-toggle-label">
- <div tabindex="0" class="version version-selector version-selector-show">
- 0.18.dev0 <span class="chevron versions-hidden"><svg fill="none" height="24" viewBox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg"><path d="m8 4 8 8-8 8" stroke="#000" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"/></svg></span><span class="chevron versions-shown"><svg fill="none" height="24" viewBox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg"><path d="m4 8 8 8 8-8" stroke="#000" stroke-linecap="round" stroke-linejoin="round [...]
- </div>
- </label>
- <div class="version-details wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
- <p class="caption" role="heading"><span class="caption-text">Versions</span></p>
- <ol style="text-align: left">
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="/">0.18.dev0 (main)</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.8.0/">v0.8.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.9.0/">v0.9.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.10.0/">v0.10.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.11.0/">v0.11.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.12.0/">v0.12.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.13.0/">v0.13.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.14.0/">v0.14.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.15.0/">v0.15.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.16.0/">v0.16.0</a></div></li>
-
- </ol>
- </div>
-
-
-
-
-<div role="search">
- <form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
- <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
- </form>
-</div>
-
-
- </div>
-
-
- <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
-
-
-
-
-
-
- <p class="caption" role="heading"><span class="caption-text">Getting Started</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/overview.html">Overview</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../install/index.html">Installing TVM</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/tutorials/quick_start.html">Quick Start</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/tutorials/ir_module.html">IRModule</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">How To</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/e2e_opt_model.html">End-to-End Optimize Model</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/customize_opt.html">Customize Optimization</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/optimize_llm.html">Optimize Large Language Model</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/cross_compilation_and_rpc.html">Cross Compilation and RPC</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/dev/index.html">Development Guides</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Deep Dive</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/tensor_ir/index.html">TensorIR</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/relax/index.html">Relax</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/python/index.html">Python API</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/links.html">Other APIs</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Legacy</span></p>
-<ul class="current">
-<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">User Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/legacy_index.html">How To Guides</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/tutorial/index.html">Developer Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/how_to/how_to.html">Developer How-To Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/langref/index.html">Language Reference</a></li>
-<li class="toctree-l1 current"><a class="reference internal" href="index.html">Design and Architecture</a><ul class="current">
-<li class="toctree-l2"><a class="reference internal" href="index.html#example-compilation-flow">Example Compilation Flow</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#logical-architecture-components">Logical Architecture Components</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-support">tvm/support</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-runtime">tvm/runtime</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-node">tvm/node</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-ir">tvm/ir</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-target">tvm/target</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-tir">tvm/tir</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-arith">tvm/arith</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-te">tvm/te</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-topi">tvm/topi</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-relay">tvm/relay</a></li>
-<li class="toctree-l2 current"><a class="reference internal" href="index.html#tvm-autotvm">tvm/autotvm</a><ul class="current">
-<li class="toctree-l3 current"><a class="current reference internal" href="#">Benchmark Performance Log Format</a><ul>
-<li class="toctree-l4"><a class="reference internal" href="#log-format-overview">Log Format Overview</a></li>
-<li class="toctree-l4"><a class="reference internal" href="#storage-format">Storage format</a></li>
-</ul>
-</li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#frontends">Frontends</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#microtvm">microTVM</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="../topic/microtvm/index.html">microTVM: TVM on bare-metal</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../topic/vta/index.html">VTA: Versatile Tensor Accelerator</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">About</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../contribute/index.html">Contributor Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/publications.html">Publications</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/security.html">Security Guide</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Index</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../genindex.html">Index</a></li>
-</ul>
-
-
-
- </div>
-
- </div>
- </nav>
-
- <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
-
- <nav class="wy-nav-top" aria-label="top navigation" data-toggle="wy-nav-top">
-
- <div class="togglemenu">
-
- </div>
- <div class="nav-content">
- <!-- tvm -->
- Table of Contents
- </div>
-
- </nav>
-
-
- <div class="wy-nav-content">
-
- <div class="rst-content">
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-<div role="navigation" aria-label="breadcrumbs navigation">
-
- <ul class="wy-breadcrumbs">
-
- <li><a href="../index.html">Docs</a> <span class="br-arrow">></span></li>
-
- <li><a href="index.html">Design and Architecture</a> <span class="br-arrow">></span></li>
-
- <li>Benchmark Performance Log Format</li>
-
-
-
-
-
-
-
-
- <li class="wy-breadcrumbs-aside">
-
-
-
- <a href="https://github.com/apache/tvm/edit/main/docs/arch/benchmark.rst" class="fa fa-github"> Edit on GitHub</a>
-
-
-
- </li>
-
- </ul>
-
-
- <hr/>
-</div>
- <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
- <div itemprop="articleBody">
-
- <div class="section" id="benchmark-performance-log-format">
-<h1>Benchmark Performance Log Format<a class="headerlink" href="#benchmark-performance-log-format" title="Permalink to this headline">¶</a></h1>
-<p>This page details schema v0.1 for a unified benchmark log format. This schema will allow easier cross-references with other frameworks/runs, experiment reproduction, data for nightly perf regression, and the separation of logging/visualization efforts.</p>
-<div class="section" id="log-format-overview">
-<h2>Log Format Overview<a class="headerlink" href="#log-format-overview" title="Permalink to this headline">¶</a></h2>
-<p>For simplicity, we suggest prioritizing the fields <cite>workload</cite>, <cite>engine</cite>, <cite>hardware</cite> <cite>runtime_ms_mean</cite>, and <cite>runtime_ms_std</cite>. For finer-grained logging, one may additionally propagate the <cite>*_config</cite> fields.</p>
-<table class="docutils align-default">
-<colgroup>
-<col style="width: 8%" />
-<col style="width: 60%" />
-<col style="width: 5%" />
-<col style="width: 27%" />
-</colgroup>
-<thead>
-<tr class="row-odd"><th class="head"><p>header</p></th>
-<th class="head"><p>examples</p></th>
-<th class="head"><p>category</p></th>
-<th class="head"><p>notes/justification</p></th>
-</tr>
-</thead>
-<tbody>
-<tr class="row-even"><td><p>workload</p></td>
-<td><p>resnet-18</p></td>
-<td><p>workload</p></td>
-<td><p>name of workload</p></td>
-</tr>
-<tr class="row-odd"><td><p>engine</p></td>
-<td><p>“tvm” / “onnxruntime”</p></td>
-<td><p>compiler</p></td>
-<td></td>
-</tr>
-<tr class="row-even"><td><p>hardware</p></td>
-<td><p>“gcp-c2-standard-16”</p></td>
-<td><p>hardware</p></td>
-<td><p>descriptor of target hardware environment</p></td>
-</tr>
-<tr class="row-odd"><td><p>runtime_ms_mean</p></td>
-<td><p>12.452</p></td>
-<td><p>statistics</p></td>
-<td></td>
-</tr>
-<tr class="row-even"><td><p>runtime_ms_std</p></td>
-<td><p>5.3</p></td>
-<td><p>statistics</p></td>
-<td></td>
-</tr>
-<tr class="row-odd"><td><p>timestamp</p></td>
-<td><p>1572282699.6</p></td>
-<td><p>metadata</p></td>
-<td><p>indicates when this record is logged</p></td>
-</tr>
-<tr class="row-even"><td><p>schema_version</p></td>
-<td><p>“0.1”</p></td>
-<td><p>metadata</p></td>
-<td><p>ensure reproducibility as we iterate on this schema</p></td>
-</tr>
-<tr class="row-odd"><td><p>metadata</p></td>
-<td><p>{ “docker_tag”:”gcr.io/…/0a680”, … }</p></td>
-<td><p>metadata</p></td>
-<td><p><code class="docutils literal notranslate"><span class="pre">docker_tag</span></code> is optional</p></td>
-</tr>
-<tr class="row-even"><td><p>workload_args</p></td>
-<td><p>{“input_name”: “Input3”, “input_shape”: [list_of_shape], “data_layout”: NHCW}</p></td>
-<td><p>workload</p></td>
-<td></td>
-</tr>
-<tr class="row-odd"><td><p>workload_metadata</p></td>
-<td><p>{“class”: “vision”,”doc_url”: “<code class="docutils literal notranslate"><span class="pre">https://github.com/.../README.md</span></code>”, “opset”: 7,”type”: “body_analysis”,”url”: “<code class="docutils literal notranslate"><span class="pre">https://onnxzoo...ferplus.tar.gz</span></code>”, “md5”: “07fc7…”}</p></td>
-<td><p>workload</p></td>
-<td><p>source of workload</p></td>
-</tr>
-<tr class="row-even"><td><p>engine_version</p></td>
-<td><p>“1.0.5”</p></td>
-<td><p>compiler</p></td>
-<td><p>use semvar format</p></td>
-</tr>
-<tr class="row-odd"><td><p>engine_config</p></td>
-<td><p>{“llvm”: “llvm-8”, “nvcc”: 10.1, “accelerator”: “MLAS”, “relay_opt_level”: 3, “tvm_target”:”llvm -mcpu=cascadelake”}</p></td>
-<td><p>compiler</p></td>
-<td><p>fields are optionally specified</p></td>
-</tr>
-<tr class="row-even"><td><p>compilation_config</p></td>
-<td><p>{“opt_level”: 3, “layer_schedules”:[]/ <SHA_to_schedules>}</p></td>
-<td><p>compiler</p></td>
-<td><p>fields are optionally specified</p></td>
-</tr>
-<tr class="row-odd"><td><p>software_config</p></td>
-<td><p>{“os”: “ubuntu:18.04”,”pip”: { “docker”: “4.1.0”, “gitpython”: “3.0.4”, “numpy”: “1.17.4”, “onnx”: “1.6.0”}, “cudnn”: “cudnn-8”, “cuda_driver”: “480.10.1”}</p></td>
-<td><p>backend</p></td>
-<td><p>env dependency list</p></td>
-</tr>
-<tr class="row-even"><td><p>runtime_config</p></td>
-<td><p>{“num_cpu_threads”: 3}</p></td>
-<td><p>backend</p></td>
-<td><p>info on non-hardware, non-software metadata</p></td>
-</tr>
-<tr class="row-odd"><td><p>hardware_config</p></td>
-<td><p>{“cpu_count”: 16, “cloud_machine_type”:”c2-standard-16”, “memory_GB”:64}</p></td>
-<td><p>hardware</p></td>
-<td><p>json descriptor of target hardware environment</p></td>
-</tr>
-<tr class="row-even"><td><p>execution_config</p></td>
-<td><p>{“number”: 1, “repeat”: 10, “min_repeat_ms”, 0}</p></td>
-<td><p>statistics</p></td>
-<td><p>workload execution parameters</p></td>
-</tr>
-<tr class="row-odd"><td><p>metrics</p></td>
-<td><p>{“accuracy”: 48.5,“compilation_ms_mean”: 12}</p></td>
-<td><p>statistics</p></td>
-<td><p>other metrics</p></td>
-</tr>
-<tr class="row-even"><td><p>runtime_raw</p></td>
-<td><p>[{“runtime_ms”: 12, …}, {“runtime_ms”:13,…},…]</p></td>
-<td><p>statistics</p></td>
-<td><p>optional raw metrics array</p></td>
-</tr>
-</tbody>
-</table>
-</div>
-<div class="section" id="storage-format">
-<h2>Storage format<a class="headerlink" href="#storage-format" title="Permalink to this headline">¶</a></h2>
-<p>Currently we’re prototyping benchmark data as JSON objects for extensibility and convenience, especially in early versions of the schema. However, as we scale up benchmark aggregation and stabilize parameters, we anticipate switching to a columnar format, such as Arrow or Parquet.</p>
-<p>Here is sample data encoded as JSON:</p>
-<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
- <span class="s2">"workload"</span><span class="p">:</span><span class="s2">"arcface_resnet100"</span><span class="p">,</span>
- <span class="s2">"engine"</span><span class="p">:</span><span class="s2">"tvm"</span><span class="p">,</span>
- <span class="s2">"hardware"</span><span class="p">:</span><span class="s2">"gcp-c2-standard-16"</span><span class="p">,</span>
- <span class="s2">"runtime_ms_mean"</span><span class="p">:</span><span class="mf">109.43004820081924</span><span class="p">,</span>
- <span class="s2">"runtime_ms_std"</span><span class="p">:</span><span class="mf">0.09078385126800587</span><span class="p">,</span>
- <span class="s2">"timestamp"</span><span class="p">:</span><span class="s2">"20191123003411"</span><span class="p">,</span>
- <span class="s2">"schema_version"</span><span class="p">:</span><span class="s2">"0.1"</span><span class="p">,</span>
- <span class="s2">"metadata"</span><span class="p">:{</span>
- <span class="s2">"docker_tag"</span><span class="p">:</span><span class="s2">"tlcpack/ci-gpu:v0.53"</span>
- <span class="p">},</span>
- <span class="s2">"workload_args"</span><span class="p">:{</span>
- <span class="s2">"input_shape_dict"</span><span class="p">:{</span>
- <span class="s2">"data"</span><span class="p">:[</span>
- <span class="mi">1</span><span class="p">,</span>
- <span class="mi">3</span><span class="p">,</span>
- <span class="mi">112</span><span class="p">,</span>
- <span class="mi">112</span>
- <span class="p">]</span>
- <span class="p">},</span>
- <span class="s2">"input_type_dict"</span><span class="p">:{</span>
- <span class="s2">"data"</span><span class="p">:</span><span class="s2">"float32"</span>
- <span class="p">},</span>
- <span class="s2">"input_value_dict"</span><span class="p">:{}</span>
- <span class="p">},</span>
- <span class="s2">"workload_metadata"</span><span class="p">:{</span>
- <span class="s2">"class"</span><span class="p">:</span><span class="s2">"vision"</span><span class="p">,</span>
- <span class="s2">"doc_url"</span><span class="p">:</span><span class="s2">"https://github.com/onnx/models/blob/main/vision/body_analysis/arcface/README.md"</span><span class="p">,</span>
- <span class="s2">"md5"</span><span class="p">:</span><span class="s2">"66074b860f905295aab5a842be57f37d"</span><span class="p">,</span>
- <span class="s2">"opset"</span><span class="p">:</span><span class="mi">8</span><span class="p">,</span>
- <span class="s2">"type"</span><span class="p">:</span><span class="s2">"body_analysis"</span><span class="p">,</span>
- <span class="s2">"url"</span><span class="p">:</span><span class="s2">"https://s3.amazonaws.com/onnx-model-zoo/arcface/resnet100/resnet100.tar.gz"</span>
- <span class="p">},</span>
- <span class="s2">"engine_version"</span><span class="p">:</span><span class="s2">"1.0.0"</span><span class="p">,</span>
- <span class="s2">"engine_config"</span><span class="p">:{},</span>
- <span class="s2">"compilation_config"</span><span class="p">:{</span>
- <span class="s2">"relay_opt_level"</span><span class="p">:</span> <span class="mi">3</span>
- <span class="p">},</span>
- <span class="s2">"software_config"</span><span class="p">:{</span>
- <span class="s2">"os"</span><span class="p">:</span><span class="s2">"ubuntu:18.04"</span><span class="p">,</span>
- <span class="s2">"pip"</span><span class="p">:{</span>
- <span class="s2">"docker"</span><span class="p">:</span><span class="s2">"4.1.0"</span><span class="p">,</span>
- <span class="s2">"gitpython"</span><span class="p">:</span><span class="s2">"3.0.4"</span><span class="p">,</span>
- <span class="s2">"numpy"</span><span class="p">:</span><span class="s2">"1.17.4"</span><span class="p">,</span>
- <span class="s2">"onnx"</span><span class="p">:</span><span class="s2">"1.6.0"</span>
- <span class="p">}</span>
- <span class="p">},</span>
- <span class="s2">"runtime_config"</span><span class="p">:{},</span>
- <span class="s2">"hardware_config"</span><span class="p">:{</span>
- <span class="s2">"cloud_machine_type"</span><span class="p">:</span><span class="s2">"c2-standard-16"</span><span class="p">,</span>
- <span class="s2">"cloud_provider"</span><span class="p">:</span><span class="s2">"GCP"</span><span class="p">,</span>
- <span class="s2">"cpu_count"</span><span class="p">:</span><span class="mi">16</span><span class="p">,</span>
- <span class="s2">"cpu_platform"</span><span class="p">:</span><span class="s2">"Intel Cascade Lake"</span><span class="p">,</span>
- <span class="s2">"memory_GB"</span><span class="p">:</span><span class="mi">64</span>
- <span class="p">},</span>
- <span class="s2">"execution_config"</span><span class="p">:{},</span>
- <span class="s2">"metrics"</span><span class="p">:{}</span>
-<span class="p">}</span>
-</pre></div>
-</div>
-</div>
-</div>
-
-
- </div>
-
- </div>
-
-
-<footer>
-
- <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
-
- <a href="frontend/tensorflow.html" class="btn btn-neutral float-right" title="TensorFlow Frontend" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
-
-
- <a href="convert_layout.html" class="btn btn-neutral float-left" title="Convert Layout Pass" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
-
- </div>
-
-<div id="button" class="backtop"><img src="../_static/img/right.svg" alt="backtop"/> </div>
-<section class="footerSec">
- <div class="footerHeader">
- <div class="d-flex align-md-items-center justify-content-between flex-column flex-md-row">
- <div class="copywrite d-flex align-items-center">
- <h5 id="copy-right-info">© 2024 Apache Software Foundation | All rights reserved</h5>
- </div>
- </div>
-
- </div>
-
- <div>
- <div class="footernote">Copyright © 2024 The Apache Software Foundation. Apache TVM, Apache, the Apache feather, and the Apache TVM project logo are either trademarks or registered trademarks of the Apache Software Foundation.</div>
- </div>
-
-</section>
-</footer>
- </div>
- </div>
-
- </section>
-
- </div>
-
-
- <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script>
- <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script>
-
- </body>
- <script type="text/javascript">
- jQuery(function () {
- SphinxRtdTheme.Navigation.enable(true);
- });
- </script>
-
-
-
-
- <!-- Theme Analytics -->
- <script>
- (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
- (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
- m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
- })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
-
- ga('create', 'UA-75982049-2', 'auto');
- ga('send', 'pageview');
- </script>
-
-
-
-
-</body>
-</html>
\ No newline at end of file
diff --git a/docs/arch/convert_layout.html b/docs/arch/convert_layout.html
deleted file mode 100644
index bfe5c9d75a..0000000000
--- a/docs/arch/convert_layout.html
+++ /dev/null
@@ -1,706 +0,0 @@
-
-
-
-
-
-
-<!DOCTYPE html>
-<html class="writer-html5" lang="en" >
-<head>
- <meta charset="utf-8">
-
- <meta name="viewport" content="width=device-width, initial-scale=1.0">
-
- <title>Convert Layout Pass — tvm 0.18.dev0 documentation</title>
-
-
-
- <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
- <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
- <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
- <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-binder.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-dataframe.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-rendered-html.css" type="text/css" />
- <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
- <link rel="stylesheet" href="../_static/css/tlcpack_theme.css" type="text/css" />
-
-
-
- <link rel="shortcut icon" href="../_static/tvm-logo-square.png"/>
-
-
-
-
-
-
-
- <script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
- <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
- <script src="../_static/jquery.js"></script>
- <script src="../_static/underscore.js"></script>
- <script src="../_static/doctools.js"></script>
-
- <script type="text/javascript" src="../_static/js/theme.js"></script>
-
-
- <script type="text/javascript" src="../_static/js/tlcpack_theme.js"></script>
- <link rel="index" title="Index" href="../genindex.html" />
- <link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Benchmark Performance Log Format" href="benchmark.html" />
- <link rel="prev" title="Relay Operator Strategy" href="relay_op_strategy.html" />
-</head>
-
-<body class="wy-body-for-nav">
-
-
- <div class="wy-grid-for-nav">
-
-
-<header class="header">
- <div class="innercontainer">
- <div class="headerInner d-flex justify-content-between align-items-center">
- <div class="headerLogo">
- <a href="https://tvm.apache.org/"><img src=https://tvm.apache.org/assets/images/logo.svg alt="logo"></a>
- </div>
-
- <div id="headMenu" class="headerNav">
- <button type="button" id="closeHeadMenu" class="navCloseBtn"><img src="../_static/img/close-icon.svg" alt="Close"></button>
- <ul class="nav">
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/community>Community</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/download>Download</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/blog>Blog</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/docs>Docs</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvmconf.org>Conference</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://github.com/apache/tvm/>Github</a>
- </li>
- </ul>
- <div class="responsivetlcdropdown">
- <button type="button" class="btn-link">
- ASF
- </button>
- <ul>
- <li>
- <a href=https://apache.org/>Apache Homepage</a>
- </li>
- <li>
- <a href=https://www.apache.org/licenses/>License</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/sponsorship.html>Sponsorship</a>
- </li>
- <li>
- <a href=https://tvm.apache.org/docs/reference/security.html>Security</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/thanks.html>Thanks</a>
- </li>
- <li>
- <a href=https://www.apache.org/events/current-event>Events</a>
- </li>
- </ul>
- </div>
- </div>
- <div class="responsiveMenuIcon">
- <button type="button" id="menuBtn" class="btn-menu"><img src="../_static/img/menu-icon.svg" alt="Menu Icon"></button>
- </div>
-
- <div class="tlcDropdown">
- <div class="dropdown">
- <button type="button" class="btn-link dropdown-toggle" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
- ASF
- </button>
- <div class="dropdown-menu dropdown-menu-right">
- <ul>
- <li>
- <a href=https://apache.org/>Apache Homepage</a>
- </li>
- <li>
- <a href=https://www.apache.org/licenses/>License</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/sponsorship.html>Sponsorship</a>
- </li>
- <li>
- <a href=https://tvm.apache.org/docs/reference/security.html>Security</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/thanks.html>Thanks</a>
- </li>
- <li>
- <a href=https://www.apache.org/events/current-event>Events</a>
- </li>
- </ul>
- </div>
- </div>
- </div>
- </div>
- </div>
- </header>
-
- <nav data-toggle="wy-nav-shift" class="wy-nav-side fixed">
- <div class="wy-side-scroll">
- <div class="wy-side-nav-search" >
-
-
-
- <a href="../index.html">
-
-
-
-
- <img src="../_static/tvm-logo-small.png" class="logo" alt="Logo"/>
-
- </a>
-
-
-
-
- <input type="checkbox" class="version-toggle-box" hidden id="version-toggle">
- <label for="version-toggle" class="version-toggle-label">
- <div tabindex="0" class="version version-selector version-selector-show">
- 0.18.dev0 <span class="chevron versions-hidden"><svg fill="none" height="24" viewBox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg"><path d="m8 4 8 8-8 8" stroke="#000" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"/></svg></span><span class="chevron versions-shown"><svg fill="none" height="24" viewBox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg"><path d="m4 8 8 8 8-8" stroke="#000" stroke-linecap="round" stroke-linejoin="round [...]
- </div>
- </label>
- <div class="version-details wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
- <p class="caption" role="heading"><span class="caption-text">Versions</span></p>
- <ol style="text-align: left">
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="/">0.18.dev0 (main)</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.8.0/">v0.8.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.9.0/">v0.9.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.10.0/">v0.10.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.11.0/">v0.11.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.12.0/">v0.12.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.13.0/">v0.13.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.14.0/">v0.14.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.15.0/">v0.15.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.16.0/">v0.16.0</a></div></li>
-
- </ol>
- </div>
-
-
-
-
-<div role="search">
- <form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
- <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
- </form>
-</div>
-
-
- </div>
-
-
- <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
-
-
-
-
-
-
- <p class="caption" role="heading"><span class="caption-text">Getting Started</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/overview.html">Overview</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../install/index.html">Installing TVM</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/tutorials/quick_start.html">Quick Start</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/tutorials/ir_module.html">IRModule</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">How To</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/e2e_opt_model.html">End-to-End Optimize Model</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/customize_opt.html">Customize Optimization</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/optimize_llm.html">Optimize Large Language Model</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/cross_compilation_and_rpc.html">Cross Compilation and RPC</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/dev/index.html">Development Guides</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Deep Dive</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/tensor_ir/index.html">TensorIR</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/relax/index.html">Relax</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/python/index.html">Python API</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/links.html">Other APIs</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Legacy</span></p>
-<ul class="current">
-<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">User Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/legacy_index.html">How To Guides</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/tutorial/index.html">Developer Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/how_to/how_to.html">Developer How-To Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/langref/index.html">Language Reference</a></li>
-<li class="toctree-l1 current"><a class="reference internal" href="index.html">Design and Architecture</a><ul class="current">
-<li class="toctree-l2"><a class="reference internal" href="index.html#example-compilation-flow">Example Compilation Flow</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#logical-architecture-components">Logical Architecture Components</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-support">tvm/support</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-runtime">tvm/runtime</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-node">tvm/node</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-ir">tvm/ir</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-target">tvm/target</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-tir">tvm/tir</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-arith">tvm/arith</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-te">tvm/te</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-topi">tvm/topi</a></li>
-<li class="toctree-l2 current"><a class="reference internal" href="index.html#tvm-relay">tvm/relay</a><ul class="current">
-<li class="toctree-l3"><a class="reference internal" href="relay_intro.html">Introduction to Relay IR</a></li>
-<li class="toctree-l3"><a class="reference internal" href="relay_op_strategy.html">Relay Operator Strategy</a></li>
-<li class="toctree-l3 current"><a class="current reference internal" href="#">Convert Layout Pass</a><ul>
-<li class="toctree-l4"><a class="reference internal" href="#background">1. Background</a></li>
-<li class="toctree-l4"><a class="reference internal" href="#motivation-and-overview">2. Motivation and Overview</a></li>
-<li class="toctree-l4"><a class="reference internal" href="#design">3. Design</a></li>
-<li class="toctree-l4"><a class="reference internal" href="#usage">4. Usage</a></li>
-</ul>
-</li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-autotvm">tvm/autotvm</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#frontends">Frontends</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#microtvm">microTVM</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="../topic/microtvm/index.html">microTVM: TVM on bare-metal</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../topic/vta/index.html">VTA: Versatile Tensor Accelerator</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">About</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../contribute/index.html">Contributor Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/publications.html">Publications</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/security.html">Security Guide</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Index</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../genindex.html">Index</a></li>
-</ul>
-
-
-
- </div>
-
- </div>
- </nav>
-
- <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
-
- <nav class="wy-nav-top" aria-label="top navigation" data-toggle="wy-nav-top">
-
- <div class="togglemenu">
-
- </div>
- <div class="nav-content">
- <!-- tvm -->
- Table of Contents
- </div>
-
- </nav>
-
-
- <div class="wy-nav-content">
-
- <div class="rst-content">
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-<div role="navigation" aria-label="breadcrumbs navigation">
-
- <ul class="wy-breadcrumbs">
-
- <li><a href="../index.html">Docs</a> <span class="br-arrow">></span></li>
-
- <li><a href="index.html">Design and Architecture</a> <span class="br-arrow">></span></li>
-
- <li>Convert Layout Pass</li>
-
-
-
-
-
-
-
-
- <li class="wy-breadcrumbs-aside">
-
-
-
- <a href="https://github.com/apache/tvm/edit/main/docs/arch/convert_layout.rst" class="fa fa-github"> Edit on GitHub</a>
-
-
-
- </li>
-
- </ul>
-
-
- <hr/>
-</div>
- <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
- <div itemprop="articleBody">
-
- <div class="section" id="convert-layout-pass">
-<h1>Convert Layout Pass<a class="headerlink" href="#convert-layout-pass" title="Permalink to this headline">¶</a></h1>
-<p><strong>Author</strong>: <a class="reference external" href="https://github.com/anijain2305">Animesh Jain</a></p>
-<div class="section" id="background">
-<h2>1. Background<a class="headerlink" href="#background" title="Permalink to this headline">¶</a></h2>
-<p>Data layout format describes how the data is laid out in the memory. For example, Tensorflow framework default data layout for convolution operator is NHWC, i.e, the data is 4-dimensions and is laid out in row-major format with N being the first dimension and C being the last dimension. Data layout has a major role in model performance, significantly affecting spatial and temporal locality. For example, Intel x86 backend in TVM prefers layout as NCHWc where the C dimension is tiled in [...]
-<p>Essentially, TVM has to deal with data layouts throughout the compiler toolchain - Framework parsers, Relay layout transformations, and TOPI schedules. As we move towards third-party codegen integration, which might have their own data layout restrictions, handling layouts at all levels in TVM toolchain is going to become even more challenging. Therefore, we developed a new Relay pass - <strong>ConvertLayout</strong> – to reduce some of the complications that arise due to layout handl [...]
-<p>If you directly want to understand the usage of ConvertLayout Pass, directly jump to Section 4 - Usage.</p>
-</div>
-<div class="section" id="motivation-and-overview">
-<h2>2. Motivation and Overview<a class="headerlink" href="#motivation-and-overview" title="Permalink to this headline">¶</a></h2>
-<p>Let’s look at a simple scenario to understand the complications that arise due to different layouts - Suppose we want to compile a Tensorflow NHWC graph for an ARM edge device. But, suppose we currently support only NCHW schedules in TOPI for ARM. So, there is a mismatch between framework layout and TOPI-supported layout. One way to deal with this mismatch is to insert layout transforms before each and after convolution, such that resulting convolution has NCHW input data layout and c [...]
-<p>We encountered similar problems in other use cases as well</p>
-<ul class="simple">
-<li><p>No way to run TFLite graphs on Nvidia GPUs. TOPI has NCHW-only schedules for GPUs.</p></li>
-<li><p>Ever-complicating logic in AlterOpLayout for convolution to support different pairs of layout transformations.</p></li>
-<li><p>Sub-optimal performance for TF graphs due to extra layout transforms.</p></li>
-<li><p>Complication in third-party codegen integrations like TensorRT that prefers data layout to be in one format.</p></li>
-</ul>
-<p>To solve these problems, we introduced <em>ConvertLayout</em> pass that sets up the infrastructure to change the data layout of the whole graph with minimal number of data layout transforms. In ideal cases, we will have only 2 layout transforms for data, one at the start and one at the end. An example to show the transformation is below</p>
-<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Original graph - 2 convolutions in NHWC format.</span>
-<span class="n">fn</span> <span class="p">(</span><span class="o">%</span><span class="n">x</span><span class="p">:</span> <span class="n">Tensor</span><span class="p">[(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">56</span><span class="p">,</span> <span class="mi">56</span><span class="p">,</span> <span class="mi">64</span><span class="p">),</span> <span class="n">float32</span><span class="p">],</span> <span class="o">%</span><span class="n">weight1</span>< [...]
- <span class="o">%</span><span class="mi">0</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">conv2d</span><span class="p">(</span><span class="o">%</span><span class="n">x</span><span class="p">,</span> <span class="o">%</span><span class="n">weight1</span><span class="p">,</span> <span class="n">padding</span><span class="o">=</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span cl [...]
- <span class="o">%</span><span class="mi">1</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">relu</span><span class="p">(</span><span class="o">%</span><span class="mi">0</span><span class="p">);</span>
- <span class="o">%</span><span class="mi">2</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">conv2d</span><span class="p">(</span><span class="o">%</span><span class="mi">1</span><span class="p">,</span> <span class="o">%</span><span class="n">weight2</span><span class="p">,</span> <span class="n">padding</span><span class="o">=</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span c [...]
- <span class="n">nn</span><span class="o">.</span><span class="n">relu</span><span class="p">(</span><span class="o">%</span><span class="mi">2</span><span class="p">)</span>
-<span class="p">}</span>
-
-<span class="c1"># After ConvertLayout - For data, there is a transform at the start and at the end.</span>
-<span class="c1"># For weights, there are transforms to adapt to NCHW layout. These will be removed by FoldConstant pass.</span>
-<span class="n">fn</span> <span class="p">(</span><span class="o">%</span><span class="n">x</span><span class="p">:</span> <span class="n">Tensor</span><span class="p">[(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">56</span><span class="p">,</span> <span class="mi">56</span><span class="p">,</span> <span class="mi">64</span><span class="p">),</span> <span class="n">float32</span><span class="p">],</span> <span class="o">%</span><span class="n">weight1</span>< [...]
- <span class="o">%</span><span class="mi">0</span> <span class="o">=</span> <span class="n">layout_transform</span><span class="p">(</span><span class="o">%</span><span class="n">x</span><span class="p">,</span> <span class="n">src_layout</span><span class="o">=</span><span class="s2">"NHWC"</span><span class="p">,</span> <span class="n">dst_layout</span><span class="o">=</span><span class="s2">"NCHW"</span><span class="p">)</span> <span class="o">/*</span> <span cla [...]
- <span class="o">%</span><span class="mi">1</span> <span class="o">=</span> <span class="n">layout_transform</span><span class="p">(</span><span class="o">%</span><span class="n">weight1</span><span class="p">,</span> <span class="n">src_layout</span><span class="o">=</span><span class="s2">"HWIO"</span><span class="p">,</span> <span class="n">dst_layout</span><span class="o">=</span><span class="s2">"OIHW"</span><span class="p">)</span> <span class="o">/*</span> <sp [...]
- <span class="o">%</span><span class="mi">2</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">conv2d</span><span class="p">(</span><span class="o">%</span><span class="mi">0</span><span class="p">,</span> <span class="o">%</span><span class="mi">1</span><span class="p">,</span> <span class="n">padding</span><span class="o">=</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class= [...]
- <span class="o">%</span><span class="mi">3</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">relu</span><span class="p">(</span><span class="o">%</span><span class="mi">2</span><span class="p">)</span> <span class="o">/*</span> <span class="n">ty</span><span class="o">=</span><span class="n">Tensor</span><span class="p">[(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class [...]
- <span class="o">%</span><span class="mi">4</span> <span class="o">=</span> <span class="n">layout_transform</span><span class="p">(</span><span class="o">%</span><span class="n">weight2</span><span class="p">,</span> <span class="n">src_layout</span><span class="o">=</span><span class="s2">"HWIO"</span><span class="p">,</span> <span class="n">dst_layout</span><span class="o">=</span><span class="s2">"OIHW"</span><span class="p">)</span> <span class="o">/*</span> <sp [...]
- <span class="o">%</span><span class="mi">5</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">conv2d</span><span class="p">(</span><span class="o">%</span><span class="mi">3</span><span class="p">,</span> <span class="o">%</span><span class="mi">4</span><span class="p">,</span> <span class="n">padding</span><span class="o">=</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class= [...]
- <span class="o">%</span><span class="mi">6</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">relu</span><span class="p">(</span><span class="o">%</span><span class="mi">5</span><span class="p">)</span> <span class="o">/*</span> <span class="n">ty</span><span class="o">=</span><span class="n">Tensor</span><span class="p">[(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class [...]
- <span class="n">layout_transform</span><span class="p">(</span><span class="o">%</span><span class="mi">6</span><span class="p">,</span> <span class="n">src_layout</span><span class="o">=</span><span class="s2">"NCHW"</span><span class="p">,</span> <span class="n">dst_layout</span><span class="o">=</span><span class="s2">"NHWC"</span><span class="p">)</span> <span class="o">/*</span> <span class="n">ty</span><span class="o">=</span><span class="n">Tensor</span><span [...]
-<span class="p">}</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="design">
-<h2>3. Design<a class="headerlink" href="#design" title="Permalink to this headline">¶</a></h2>
-<p>Before delving into ConvertLayout pass, let’s categorize the operators into 3 categories based on their sensitivity to data layouts. This categorization will be useful later to understand Convertlayout pass details.</p>
-<ul class="simple">
-<li><p><strong>Layout agnostic</strong> - Relu, pow etc. These operators are not affected, neither functionality nor performance, by data layouts.</p></li>
-<li><p><strong>Lightly-layout sensitive</strong> - pad, concatenate, reduce ops like sum etc. These operators have some attributes that are functionally affected if we do a layout transformation before them. However, performance-wise, the difference is not significant. For these operators, it is beneficial to just adapt to the previous operator output data layout.</p></li>
-<li><p><strong>Heavily-layout sensitive</strong> - Convolution, conv2d_transpose etc. These operators are heavily affected, both functionally and performance-wise, by data layouts. They also have data layout as the op attribute. Typically, it is beneficial to modify the input data layouts for these operators (if its not a performant data layout), while the rest of <em>layout agnostic</em> and <em>lightly-layout sensitive</em> operators adapt to the layout governed by the output of these [...]
-</ul>
-<p>Let us now look at two relevant Relay operator properties. Each relay operator has properties, like InferType, that can be defined by a TVM developer. Typically, a Relay pass traverses the graph operator-by-operator and reads these operator properties. For example, InferType pass looks at the InferType property of on operator, determines its output shape and type, and then passes it to the next operator InferType property. Similarly, in our context, we have 2 such properties - <em>FTV [...]
-<ul class="simple">
-<li><p>Run FTVMConvertLayout property - This allows the developers to transform the original Relay expr into a new Relay expr with new layouts, allowing user-defined layout alteration. There is a python callback for developer’s ease. This is used only for heavily-layout sensitive operators.</p></li>
-<li><p>Run FTVMInferCorretLayout property - We can view this as layout inference. It looks at the original input layout and the new input layouts, which are either coming from previous operator or from the FTVMConvertLayout modified expr (if it was used). This can be used by lightly-layout sensitive operators to adapt its attributes to new data layouts. Layout inference happens for each operator.</p></li>
-<li><p>Automatic insertion of layout transforms - The previous step - layout inference - sets the new layout for the input exprs. If these layouts are different from the original layouts, then this component automatically inserts a layout transform. Therefore, a developer does not need to do anything for this component.</p></li>
-</ul>
-<p>These steps happen for each operator in sequence, where ConvertLayout pass keeps on passing the new layouts to the next operator properties, finally resulting in modifying the whole graph operator-by-operator. Now, let’s look at a couple of examples of how to define the two properties.</p>
-<p><strong>FTVMConvertLayout - Python callback for layout alteration</strong> - This is used for <em>heavily-layout sensitive</em> operators. For example, one can return a new convolution operator with new data and kernel layout. The other 2 components will infer layout and insert layout transforms if needed. One example for convolution operator is as follows where we are converting to NCHW layout.</p>
-<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="nd">@reg</span><span class="o">.</span><span class="n">register_convert_op_layout</span><span class="p">(</span><span class="s2">"nn.conv2d"</span><span class="p">)</span>
-<span class="k">def</span> <span class="nf">convert_conv2d</span><span class="p">(</span><span class="n">attrs</span><span class="p">,</span> <span class="n">inputs</span><span class="p">,</span> <span class="n">tinfos</span><span class="p">,</span> <span class="n">desired_layouts</span><span class="p">):</span>
-<span class="w"> </span><span class="sd">"""Convert Layout pass registration for conv2d op.</span>
-
-<span class="sd"> Parameters</span>
-<span class="sd"> ----------</span>
-<span class="sd"> attrs : tvm.attrs.Attrs</span>
-<span class="sd"> Attributes of current convolution</span>
-<span class="sd"> inputs : list of tvm.relay.Expr</span>
-<span class="sd"> The args of the Relay expr to be legalized</span>
-<span class="sd"> tinfos : list of types</span>
-<span class="sd"> List of input and output types</span>
-<span class="sd"> desired_layouts : list of layout strings</span>
-<span class="sd"> List of layouts defining our desired</span>
-<span class="sd"> layout for the data and kernel inputs respectively.</span>
-
-<span class="sd"> Returns</span>
-<span class="sd"> -------</span>
-<span class="sd"> result : tvm.relay.Expr</span>
-<span class="sd"> The transformed expr</span>
-<span class="sd"> """</span>
-
- <span class="kn">from</span> <span class="nn">tvm</span> <span class="kn">import</span> <span class="n">relay</span>
- <span class="n">data</span><span class="p">,</span> <span class="n">weight</span> <span class="o">=</span> <span class="n">inputs</span>
- <span class="n">new_attrs</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">attrs</span><span class="p">)</span>
-
- <span class="c1"># We expect 2 desired layouts to be specified, one for the data and one for the kernel.</span>
- <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">desired_layouts</span><span class="p">)</span> <span class="o">==</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">"A desired layout is expected for both of nn.conv2d's inputs"</span>
-
- <span class="c1"># Use the first entry in desired layouts which specifies the data layout.</span>
- <span class="c1"># The expected ordering of layouts for this operator is defined by this function.</span>
- <span class="n">desired_data_layout</span><span class="p">,</span> <span class="n">desired_kernel_layout</span> <span class="o">=</span> <span class="nb">map</span><span class="p">(</span><span class="nb">str</span><span class="p">,</span> <span class="n">desired_layouts</span><span class="p">)</span>
-
- <span class="k">assert</span> <span class="n">desired_data_layout</span> <span class="o">!=</span> <span class="s2">"default"</span><span class="p">,</span> <span class="s2">"Data layout cannot be default"</span>
-
- <span class="n">new_attrs</span><span class="p">[</span><span class="s1">'data_layout'</span><span class="p">]</span> <span class="o">=</span> <span class="n">desired_data_layout</span>
-
- <span class="k">if</span> <span class="n">desired_data_layout</span> <span class="o">==</span> <span class="s1">'NCHW'</span><span class="p">:</span>
- <span class="k">if</span> <span class="n">desired_kernel_layout</span> <span class="o">!=</span> <span class="s1">'default'</span><span class="p">:</span>
- <span class="n">new_attrs</span><span class="p">[</span><span class="s1">'kernel_layout'</span><span class="p">]</span> <span class="o">=</span> <span class="n">desired_kernel_layout</span>
- <span class="k">else</span><span class="p">:</span>
- <span class="n">new_attrs</span><span class="p">[</span><span class="s1">'kernel_layout'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'OIHW'</span>
- <span class="c1"># Actual insertion of layout transforms is taken care internally</span>
- <span class="c1"># by ConvertLayout pass.</span>
- <span class="k">return</span> <span class="n">relay</span><span class="o">.</span><span class="n">nn</span><span class="o">.</span><span class="n">conv2d</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">weight</span><span class="p">,</span> <span class="o">**</span><span class="n">new_attrs</span><span class="p">)</span>
-
- <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s1">'Layout </span><span class="si">%s</span><span class="s1"> is not yet supported'</span> <span class="o">%</span> <span class="n">desired_data_layout</span><span class="p">)</span>
-</pre></div>
-</div>
-<p><strong>FInferCorrectLayout - Layout inference</strong> - Currently, this attribute is exposed only in C++. This function takes original input layouts and the new input layouts (passed from the previous operator or from the python callback for layout alteration), and infers the final data layouts. Layout inference is called for each operator. The usage might vary for different operator categories. For layout agnostic operators, we just want to return the new data layouts in this funct [...]
-<p>First example is for layout agnostic operators. These operators do not have any operator attributes that are affected by data layouts, so we just adapt to new layouts.</p>
-<div class="highlight-c++ notranslate"><div class="highlight"><pre><span></span><span class="c1">// For operator set its attributes like following</span>
-<span class="c1">// .set_attr<FInferCorrectLayout>("FInferCorrectLayout", ElemwiseArbitraryLayout);</span>
-
-<span class="c1">// Take arbitrary input layouts and copy to outputs.</span>
-<span class="kr">inline</span><span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Array</span><span class="o"><</span><span class="n">Layout</span><span class="o">>></span><span class="w"> </span><span class="n">ElemwiseArbitraryLayout</span><span class="p">(</span><span class="k">const</span><span class="w"> </span><span class="n">Attrs</span><span class="o">&</span><span class="w"> </span><span class="n">attrs</span><span class="p [...]
-<span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Layout</span><span class="o">>&</span><span class="w"> </span><span class="n">new_in_layouts</span><span class="p">,</span>
-<span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Layout</span><span class="o">>&</span><span class="w"> </span><span class="n">old_in_layouts</span><span class="p">,</span>
-<span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Array</span><span class="o"><</span><span class="n">IndexExpr</span><span class="o">>></span><span class="w"> </span><span class="o">&</span><span class="n">old_in_shapes</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="n">Layout</span><span class="w"> </span><span class="n">ret</span><span class="p">;</span>
-
-<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">new_in_layouts</span><span class="p">.</span><span class="n">defined</span><span class="p">())</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="n">ICHECK_GE</span><span class="p">(</span><span class="n">new_in_layouts</span><span class="p">.</span><span class="n">size</span><span class="p">(),</span><span class="w"> </span><span class="mi">1</span><span class="p">);</span>
-<span class="w"> </span><span class="n">ret</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">new_in_layouts</span><span class="p">[</span><span class="mi">0</span><span class="p">];</span>
-<span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="kt">size_t</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="n">old_in_layouts</span><span class="p">.</span><span cl [...]
-<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">old_in_layouts</span><span class="p">[</span><span class="n">i</span><span class="p">].</span><span class="n">defined</span><span class="p">())</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="n">ret</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">old_in_layouts</span><span class="p">[</span><span class="n">i</span><span class="p">];</span>
-<span class="w"> </span><span class="k">break</span><span class="p">;</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-
-<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Array</span><span class="o"><</span><span class="n">Layout</span><span class="o">>></span><span class="p">{</span><span class="n">Array</span><span class="o"><</span><span class="n">Layout</span><span class="o">></span><span class="p">(</span><span class="n">old_in_layouts</span><span class="p">.</span><span class="n">size</sp [...]
-<span class="p">}</span>
-</pre></div>
-</div>
-<p>Second example is for a lightly-layout sensitive operator - batch normalization. BatchNorm has an axis operator that has to change when we go from NHWC to NCHW data layout. (Similar handling also needs to be for heavily-layout sensitive operators)</p>
-<div class="highlight-c++ notranslate"><div class="highlight"><pre><span></span><span class="n">Array</span><span class="o"><</span><span class="n">Array</span><span class="o"><</span><span class="n">Layout</span><span class="o">>></span><span class="w"> </span><span class="n">BatchNormInferCorrectLayout</span><span class="p">(</span><span class="k">const</span><span class="w"> </span><span class="n">Attrs</span><span class="o">&</span><span class="w"> </span><span class= [...]
-<span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Layout</span><span class="o">>&</span><span class="w"> </span><span class="n">new_in_layouts</span><span class="p">,</span>
-<span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Layout</span><span class="o">>&</span><span class="w"> </span><span class="n">old_in_layouts</span><span class="p">,</span>
-<span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Array</span><span class="o"><</span><span class="n">IndexExpr</span><span class="o">>>&</span><span class="w"> </span><span class="n">old_in_shapes</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="n">BatchNormAttrs</span><span class="o">*</span><span class="w"> </span><span class="n">param</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">const_cast</span><span class="o"><</span><span class="n">BatchNormAttrs</span><span class="o">*></span><span class="p">(</span><span class="n">attrs</span><span class="p">.</span><span class="n">as</span><span class="o"><</span><span class="n">BatchNormAttr [...]
-
-<span class="w"> </span><span class="kt">size_t</span><span class="w"> </span><span class="n">axis</span><span class="w"> </span><span class="o">=</span>
-<span class="w"> </span><span class="n">param</span><span class="o">-></span><span class="n">axis</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="o">?</span><span class="w"> </span><span class="n">param</span><span class="o">-></span><span class="n">axis</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">old_in_shapes</span><span class="p">[ [...]
-
-<span class="w"> </span><span class="n">Layout</span><span class="w"> </span><span class="n">ret</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Layout</span><span class="o">::</span><span class="n">Undef</span><span class="p">();</span>
-
-<span class="w"> </span><span class="c1">// For example, consider old_layout = NHWC, and new_layout = NCHW, and param->axis = 3</span>
-
-<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">new_in_layouts</span><span class="p">.</span><span class="n">defined</span><span class="p">()</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span><span class="n">old_in_layouts</span><span class="p">.</span><span class="n">defined</span><span class="p">())</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="c1">// Get the new C axis. Extract the dim in old layout. Find the index of that dim in next layout.</span>
-
-<span class="w"> </span><span class="c1">// Following line gives bn_dim = C as old_layout = NHWC, axis = 3</span>
-<span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="k">auto</span><span class="o">&</span><span class="w"> </span><span class="n">bn_dim</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">old_in_layouts</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="n">axis</span><span class="p">];</span>
-
-<span class="w"> </span><span class="c1">// The new_index is 1 because new_layout = NCHW and bn_dim is C</span>
-<span class="w"> </span><span class="k">auto</span><span class="w"> </span><span class="n">new_index</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">new_in_layouts</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">IndexOf</span><span class="p">(</span><span class="n">bn_dim</span><span class="p">);</span>
-
-<span class="w"> </span><span class="c1">// We modify the layout-dependent attribute here - axis to 1.</span>
-<span class="w"> </span><span class="n">param</span><span class="o">-></span><span class="n">axis</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">new_index</span><span class="p">;</span>
-
-<span class="w"> </span><span class="c1">// Finally, we adapt to the new layout.</span>
-<span class="w"> </span><span class="n">ret</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">new_in_layouts</span><span class="p">[</span><span class="mi">0</span><span class="p">];</span>
-
-<span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">old_in_layouts</span><span class="p">.</span><span class="n">defined</span><span class="p">())</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="n">ret</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">old_in_layouts</span><span class="p">[</span><span class="mi">0</span><span class="p">];</span>
-<span class="w"> </span><span class="p">}</span>
-
-<span class="w"> </span><span class="c1">// In case both new and old layouts are undefined, then there is no need of a change.</span>
-<span class="w"> </span><span class="c1">// ConvertLayout pass skips the automatic insertion of layout transforms in this case.</span>
-
-<span class="w"> </span><span class="c1">// Following line is not important to tutorial. But, layout inference needs to define</span>
-<span class="w"> </span><span class="c1">// the layout for all input and output data layouts. For batch norm, the other inputs</span>
-<span class="w"> </span><span class="c1">// and outputs are vector having length of C dim in the input. So, we set the other</span>
-<span class="w"> </span><span class="c1">// layouts as C. BN has 5 inputs, 3 outputs. The last 4 inputs and last 2 outputs</span>
-<span class="w"> </span><span class="c1">// have "C" layout.</span>
-<span class="w"> </span><span class="n">Layout</span><span class="w"> </span><span class="n">c_layout</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Layout</span><span class="p">(</span><span class="s">"C"</span><span class="p">);</span>
-
-<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Array</span><span class="o"><</span><span class="n">Layout</span><span class="o">>></span><span class="p">{{</span><span class="n">ret</span><span class="p">,</span><span class="w"> </span><span class="n">c_layout</span><span class="p">,</span><span class="w"> </span><span class="n">c_layout</span><span class="p">,</span><span class [...]
-<span class="w"> </span><span class="p">{</span><span class="n">ret</span><span class="p">,</span><span class="w"> </span><span class="n">c_layout</span><span class="p">,</span><span class="w"> </span><span class="n">c_layout</span><span class="p">}};</span>
-<span class="p">}</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="usage">
-<h2>4. Usage<a class="headerlink" href="#usage" title="Permalink to this headline">¶</a></h2>
-<p id="convert-layout-usage">ConvertLayout pass is extremely easy to use. The pass is not a part of default relay.build pipeline. The intended usage is to call it between the framework-to-relay parser and relay.build module call.</p>
-<p>In order to specify the layouts to convert to, we create a mapping of heavily-layout sensitive operators to a list of the desired layouts for that operator. The first example below specifies data layout, we allow the kernel layout to be automatically converted to one that is supported by TVM (for that particular data layout and operator). This is specified by the use of the “default” keyword. The second example shows how we could have also converted to a specific kernel layout of our [...]
-<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># TFlite framework to Relay parser - Default layout is NHWC</span>
-<span class="n">mod</span><span class="p">,</span> <span class="n">params</span> <span class="o">=</span> <span class="n">relay</span><span class="o">.</span><span class="n">frontend</span><span class="o">.</span><span class="n">from_tflite</span><span class="p">(</span><span class="n">tflite_model</span><span class="p">,</span>
- <span class="n">shape_dict</span><span class="o">=</span><span class="n">shape_dict</span><span class="p">,</span>
- <span class="n">dtype_dict</span><span class="o">=</span><span class="n">dtype_dict</span><span class="p">)</span>
-
-<span class="c1"># We assume our model's heavily-layout sensitive operators only consist of nn.conv2d</span>
-<span class="n">desired_layouts</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'nn.conv2d'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'NCHW'</span><span class="p">,</span> <span class="s1">'default'</span><span class="p">]}</span>
-
-<span class="c1"># Convert the layout to NCHW</span>
-<span class="c1"># RemoveUnunsedFunctions is used to clean up the graph.</span>
-<span class="n">seq</span> <span class="o">=</span> <span class="n">tvm</span><span class="o">.</span><span class="n">transform</span><span class="o">.</span><span class="n">Sequential</span><span class="p">([</span><span class="n">relay</span><span class="o">.</span><span class="n">transform</span><span class="o">.</span><span class="n">RemoveUnusedFunctions</span><span class="p">(),</span>
- <span class="n">relay</span><span class="o">.</span><span class="n">transform</span><span class="o">.</span><span class="n">ConvertLayout</span><span class="p">(</span><span class="n">desired_layouts</span><span class="p">)])</span>
-<span class="k">with</span> <span class="n">tvm</span><span class="o">.</span><span class="n">transform</span><span class="o">.</span><span class="n">PassContext</span><span class="p">(</span><span class="n">opt_level</span><span class="o">=</span><span class="mi">3</span><span class="p">):</span>
- <span class="n">mod</span> <span class="o">=</span> <span class="n">seq</span><span class="p">(</span><span class="n">mod</span><span class="p">)</span>
-
-<span class="c1"># Call relay compilation</span>
-<span class="k">with</span> <span class="n">relay</span><span class="o">.</span><span class="n">build_config</span><span class="p">(</span><span class="n">opt_level</span><span class="o">=</span><span class="mi">3</span><span class="p">):</span>
- <span class="n">graph</span><span class="p">,</span> <span class="n">lib</span><span class="p">,</span> <span class="n">params</span> <span class="o">=</span> <span class="n">relay</span><span class="o">.</span><span class="n">build</span><span class="p">(</span><span class="n">mod</span><span class="p">,</span> <span class="n">target</span><span class="p">,</span> <span class="n">params</span><span class="o">=</span><span class="n">params</span><span class="p">)</span>
-</pre></div>
-</div>
-<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">desired_layouts</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'nn.conv2d'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'NCHW'</span><span class="p">,</span> <span class="s1">'OIHW'</span><span class="p">]}</span>
-<span class="k">pass</span> <span class="o">=</span> <span class="n">relay</span><span class="o">.</span><span class="n">transform</span><span class="o">.</span><span class="n">ConvertLayout</span><span class="p">(</span><span class="n">desired_layouts</span><span class="p">)</span>
-</pre></div>
-</div>
-<p>The ordering of the layouts is defined by the implementation of <cite>register_convert_op_layout(“OPNAME”)</cite>, you can refer to the docstring which should explicitly state the expected layout. In the examples above it’s [data_layout, kernel_layout].</p>
-<p>Current implementation has support for almost all the operators commonly used in image classification models. However, if one encounters too many data layout transforms in the graph, it is highly likely that there is an operator whose layouts need special handling as described in Section 3. Some pull requests that can help in such a situation are</p>
-<ul class="simple">
-<li><p>Layout inference for <a class="reference external" href="https://github.com/apache/tvm/pull/4600">Batch Norm</a> - Batch normalization falls into the category of lightly-sensitive operator. The PR shows how to handle the layout inference for batch norm.</p></li>
-<li><p>Python Callback for <a class="reference external" href="https://github.com/apache/tvm/pull/4335">Convolution</a>- For highly-sensitive operators, one might have to do python callback as well. The PR shows how to define a python callback function for Convolution operator.</p></li>
-</ul>
-</div>
-</div>
-
-
- </div>
-
- </div>
-
-
-<footer>
-
- <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
-
- <a href="benchmark.html" class="btn btn-neutral float-right" title="Benchmark Performance Log Format" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
-
-
- <a href="relay_op_strategy.html" class="btn btn-neutral float-left" title="Relay Operator Strategy" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
-
- </div>
-
-<div id="button" class="backtop"><img src="../_static/img/right.svg" alt="backtop"/> </div>
-<section class="footerSec">
- <div class="footerHeader">
- <div class="d-flex align-md-items-center justify-content-between flex-column flex-md-row">
- <div class="copywrite d-flex align-items-center">
- <h5 id="copy-right-info">© 2024 Apache Software Foundation | All rights reserved</h5>
- </div>
- </div>
-
- </div>
-
- <div>
- <div class="footernote">Copyright © 2024 The Apache Software Foundation. Apache TVM, Apache, the Apache feather, and the Apache TVM project logo are either trademarks or registered trademarks of the Apache Software Foundation.</div>
- </div>
-
-</section>
-</footer>
- </div>
- </div>
-
- </section>
-
- </div>
-
-
- <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script>
- <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script>
-
- </body>
- <script type="text/javascript">
- jQuery(function () {
- SphinxRtdTheme.Navigation.enable(true);
- });
- </script>
-
-
-
-
- <!-- Theme Analytics -->
- <script>
- (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
- (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
- m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
- })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
-
- ga('create', 'UA-75982049-2', 'auto');
- ga('send', 'pageview');
- </script>
-
-
-
-
-</body>
-</html>
\ No newline at end of file
diff --git a/docs/arch/debugger.html b/docs/arch/debugger.html
index 8dc15e2f42..8c5f093a23 100644
--- a/docs/arch/debugger.html
+++ b/docs/arch/debugger.html
@@ -48,7 +48,7 @@
<script type="text/javascript" src="../_static/js/tlcpack_theme.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Putting the VM in TVM: The Relay Virtual Machine" href="virtual_machine.html" />
+ <link rel="next" title="Introduction to Module Serialization" href="introduction_to_module_serialization.html" />
<link rel="prev" title="Vulkan Runtime" href="runtimes/vulkan.html" />
</head>
@@ -269,25 +269,9 @@
<li class="toctree-l1"><a class="reference internal" href="../how_to/dev/index.html">Development Guides</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Deep Dive</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/tensor_ir/index.html">TensorIR</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/relax/index.html">Relax</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/python/index.html">Python API</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/links.html">Other APIs</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Legacy</span></p>
<ul class="current">
-<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">User Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/legacy_index.html">How To Guides</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/tutorial/index.html">Developer Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/how_to/how_to.html">Developer How-To Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/langref/index.html">Language Reference</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="index.html">Design and Architecture</a><ul class="current">
-<li class="toctree-l2"><a class="reference internal" href="index.html#example-compilation-flow">Example Compilation Flow</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#logical-architecture-components">Logical Architecture Components</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index.html#overall-flow">Overall Flow</a></li>
<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-support">tvm/support</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="index.html#tvm-runtime">tvm/runtime</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="runtime.html">TVM Runtime System</a></li>
@@ -298,7 +282,6 @@
<li class="toctree-l4"><a class="reference internal" href="#sample-output">Sample Output</a></li>
</ul>
</li>
-<li class="toctree-l3"><a class="reference internal" href="virtual_machine.html">Putting the VM in TVM: The Relay Virtual Machine</a></li>
<li class="toctree-l3"><a class="reference internal" href="introduction_to_module_serialization.html">Introduction to Module Serialization</a></li>
<li class="toctree-l3"><a class="reference internal" href="device_target_interactions.html">Device/Target Interactions</a></li>
</ul>
@@ -306,16 +289,29 @@
<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-node">tvm/node</a></li>
<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-ir">tvm/ir</a></li>
<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-target">tvm/target</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-relax">tvm/relax</a></li>
<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-tir">tvm/tir</a></li>
<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-arith">tvm/arith</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-te">tvm/te</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-topi">tvm/topi</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-relay">tvm/relay</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-autotvm">tvm/autotvm</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#frontends">Frontends</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#microtvm">microTVM</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-te-and-tvm-topi">tvm/te and tvm/topi</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-meta-schedule">tvm/meta_schedule</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-dlight">tvm/dlight</a></li>
</ul>
</li>
+<li class="toctree-l1"><a class="reference internal" href="../deep_dive/tensor_ir/index.html">TensorIR</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../deep_dive/relax/index.html">Relax</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../reference/api/python/index.html">Python API</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../reference/api/links.html">Other APIs</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Legacy</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">User Tutorial</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../how_to/legacy_index.html">How To Guides</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../dev/tutorial/index.html">Developer Tutorial</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../dev/how_to/how_to.html">Developer How-To Guide</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../reference/langref/index.html">Language Reference</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topic/microtvm/index.html">microTVM: TVM on bare-metal</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topic/vta/index.html">VTA: Versatile Tensor Accelerator</a></li>
</ul>
@@ -593,7 +589,7 @@ folder specified while creating the runtime.</p>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
- <a href="virtual_machine.html" class="btn btn-neutral float-right" title="Putting the VM in TVM: The Relay Virtual Machine" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
+ <a href="introduction_to_module_serialization.html" class="btn btn-neutral float-right" title="Introduction to Module Serialization" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="runtimes/vulkan.html" class="btn btn-neutral float-left" title="Vulkan Runtime" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
diff --git a/docs/arch/device_target_interactions.html b/docs/arch/device_target_interactions.html
index d56a9f194a..57de8443ce 100644
--- a/docs/arch/device_target_interactions.html
+++ b/docs/arch/device_target_interactions.html
@@ -269,31 +269,14 @@
<li class="toctree-l1"><a class="reference internal" href="../how_to/dev/index.html">Development Guides</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Deep Dive</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/tensor_ir/index.html">TensorIR</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/relax/index.html">Relax</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/python/index.html">Python API</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/links.html">Other APIs</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Legacy</span></p>
<ul class="current">
-<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">User Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/legacy_index.html">How To Guides</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/tutorial/index.html">Developer Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/how_to/how_to.html">Developer How-To Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/langref/index.html">Language Reference</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="index.html">Design and Architecture</a><ul class="current">
-<li class="toctree-l2"><a class="reference internal" href="index.html#example-compilation-flow">Example Compilation Flow</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#logical-architecture-components">Logical Architecture Components</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index.html#overall-flow">Overall Flow</a></li>
<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-support">tvm/support</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="index.html#tvm-runtime">tvm/runtime</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="runtime.html">TVM Runtime System</a></li>
<li class="toctree-l3"><a class="reference internal" href="runtime.html#runtime-specific-information">Runtime-Specific Information</a></li>
<li class="toctree-l3"><a class="reference internal" href="debugger.html">Debugger</a></li>
-<li class="toctree-l3"><a class="reference internal" href="virtual_machine.html">Putting the VM in TVM: The Relay Virtual Machine</a></li>
<li class="toctree-l3"><a class="reference internal" href="introduction_to_module_serialization.html">Introduction to Module Serialization</a></li>
<li class="toctree-l3 current"><a class="current reference internal" href="#">Device/Target Interactions</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#deviceapi">DeviceAPI</a></li>
@@ -314,16 +297,29 @@
</li>
</ul>
</li>
+<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-relax">tvm/relax</a></li>
<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-tir">tvm/tir</a></li>
<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-arith">tvm/arith</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-te">tvm/te</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-topi">tvm/topi</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-relay">tvm/relay</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-autotvm">tvm/autotvm</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#frontends">Frontends</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#microtvm">microTVM</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-te-and-tvm-topi">tvm/te and tvm/topi</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-meta-schedule">tvm/meta_schedule</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-dlight">tvm/dlight</a></li>
</ul>
</li>
+<li class="toctree-l1"><a class="reference internal" href="../deep_dive/tensor_ir/index.html">TensorIR</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../deep_dive/relax/index.html">Relax</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../reference/api/python/index.html">Python API</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../reference/api/links.html">Other APIs</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Legacy</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">User Tutorial</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../how_to/legacy_index.html">How To Guides</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../dev/tutorial/index.html">Developer Tutorial</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../dev/how_to/how_to.html">Developer How-To Guide</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../reference/langref/index.html">Language Reference</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topic/microtvm/index.html">microTVM: TVM on bare-metal</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topic/vta/index.html">VTA: Versatile Tensor Accelerator</a></li>
</ul>
diff --git a/docs/arch/frontend/tensorflow.html b/docs/arch/frontend/tensorflow.html
deleted file mode 100644
index ab7de7667a..0000000000
--- a/docs/arch/frontend/tensorflow.html
+++ /dev/null
@@ -1,709 +0,0 @@
-
-
-
-
-
-
-<!DOCTYPE html>
-<html class="writer-html5" lang="en" >
-<head>
- <meta charset="utf-8">
-
- <meta name="viewport" content="width=device-width, initial-scale=1.0">
-
- <title>TensorFlow Frontend — tvm 0.18.dev0 documentation</title>
-
-
-
- <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
- <link rel="stylesheet" href="../../_static/css/theme.css" type="text/css" />
- <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
- <link rel="stylesheet" href="../../_static/css/theme.css" type="text/css" />
- <link rel="stylesheet" href="../../_static/sg_gallery.css" type="text/css" />
- <link rel="stylesheet" href="../../_static/sg_gallery-binder.css" type="text/css" />
- <link rel="stylesheet" href="../../_static/sg_gallery-dataframe.css" type="text/css" />
- <link rel="stylesheet" href="../../_static/sg_gallery-rendered-html.css" type="text/css" />
- <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
- <link rel="stylesheet" href="../../_static/css/tlcpack_theme.css" type="text/css" />
-
-
-
- <link rel="shortcut icon" href="../../_static/tvm-logo-square.png"/>
-
-
-
-
-
-
-
- <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
- <script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
- <script src="../../_static/jquery.js"></script>
- <script src="../../_static/underscore.js"></script>
- <script src="../../_static/doctools.js"></script>
-
- <script type="text/javascript" src="../../_static/js/theme.js"></script>
-
-
- <script type="text/javascript" src="../../_static/js/tlcpack_theme.js"></script>
- <link rel="index" title="Index" href="../../genindex.html" />
- <link rel="search" title="Search" href="../../search.html" />
- <link rel="next" title="microTVM Design Document" href="../microtvm_design.html" />
- <link rel="prev" title="Benchmark Performance Log Format" href="../benchmark.html" />
-</head>
-
-<body class="wy-body-for-nav">
-
-
- <div class="wy-grid-for-nav">
-
-
-<header class="header">
- <div class="innercontainer">
- <div class="headerInner d-flex justify-content-between align-items-center">
- <div class="headerLogo">
- <a href="https://tvm.apache.org/"><img src=https://tvm.apache.org/assets/images/logo.svg alt="logo"></a>
- </div>
-
- <div id="headMenu" class="headerNav">
- <button type="button" id="closeHeadMenu" class="navCloseBtn"><img src="../../_static/img/close-icon.svg" alt="Close"></button>
- <ul class="nav">
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/community>Community</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/download>Download</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/blog>Blog</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/docs>Docs</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvmconf.org>Conference</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://github.com/apache/tvm/>Github</a>
- </li>
- </ul>
- <div class="responsivetlcdropdown">
- <button type="button" class="btn-link">
- ASF
- </button>
- <ul>
- <li>
- <a href=https://apache.org/>Apache Homepage</a>
- </li>
- <li>
- <a href=https://www.apache.org/licenses/>License</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/sponsorship.html>Sponsorship</a>
- </li>
- <li>
- <a href=https://tvm.apache.org/docs/reference/security.html>Security</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/thanks.html>Thanks</a>
- </li>
- <li>
- <a href=https://www.apache.org/events/current-event>Events</a>
- </li>
- </ul>
- </div>
- </div>
- <div class="responsiveMenuIcon">
- <button type="button" id="menuBtn" class="btn-menu"><img src="../../_static/img/menu-icon.svg" alt="Menu Icon"></button>
- </div>
-
- <div class="tlcDropdown">
- <div class="dropdown">
- <button type="button" class="btn-link dropdown-toggle" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
- ASF
- </button>
- <div class="dropdown-menu dropdown-menu-right">
- <ul>
- <li>
- <a href=https://apache.org/>Apache Homepage</a>
- </li>
- <li>
- <a href=https://www.apache.org/licenses/>License</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/sponsorship.html>Sponsorship</a>
- </li>
- <li>
- <a href=https://tvm.apache.org/docs/reference/security.html>Security</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/thanks.html>Thanks</a>
- </li>
- <li>
- <a href=https://www.apache.org/events/current-event>Events</a>
- </li>
- </ul>
- </div>
- </div>
- </div>
- </div>
- </div>
- </header>
-
- <nav data-toggle="wy-nav-shift" class="wy-nav-side fixed">
- <div class="wy-side-scroll">
- <div class="wy-side-nav-search" >
-
-
-
- <a href="../../index.html">
-
-
-
-
- <img src="../../_static/tvm-logo-small.png" class="logo" alt="Logo"/>
-
- </a>
-
-
-
-
- <input type="checkbox" class="version-toggle-box" hidden id="version-toggle">
- <label for="version-toggle" class="version-toggle-label">
- <div tabindex="0" class="version version-selector version-selector-show">
- 0.18.dev0 <span class="chevron versions-hidden"><svg fill="none" height="24" viewBox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg"><path d="m8 4 8 8-8 8" stroke="#000" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"/></svg></span><span class="chevron versions-shown"><svg fill="none" height="24" viewBox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg"><path d="m4 8 8 8 8-8" stroke="#000" stroke-linecap="round" stroke-linejoin="round [...]
- </div>
- </label>
- <div class="version-details wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
- <p class="caption" role="heading"><span class="caption-text">Versions</span></p>
- <ol style="text-align: left">
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="/">0.18.dev0 (main)</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.8.0/">v0.8.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.9.0/">v0.9.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.10.0/">v0.10.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.11.0/">v0.11.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.12.0/">v0.12.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.13.0/">v0.13.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.14.0/">v0.14.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.15.0/">v0.15.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.16.0/">v0.16.0</a></div></li>
-
- </ol>
- </div>
-
-
-
-
-<div role="search">
- <form id="rtd-search-form" class="wy-form" action="../../search.html" method="get">
- <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
- </form>
-</div>
-
-
- </div>
-
-
- <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
-
-
-
-
-
-
- <p class="caption" role="heading"><span class="caption-text">Getting Started</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../../get_started/overview.html">Overview</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../install/index.html">Installing TVM</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../get_started/tutorials/quick_start.html">Quick Start</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../get_started/tutorials/ir_module.html">IRModule</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">How To</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../../how_to/tutorials/e2e_opt_model.html">End-to-End Optimize Model</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../how_to/tutorials/customize_opt.html">Customize Optimization</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../how_to/tutorials/optimize_llm.html">Optimize Large Language Model</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../how_to/tutorials/cross_compilation_and_rpc.html">Cross Compilation and RPC</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../how_to/dev/index.html">Development Guides</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Deep Dive</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../../deep_dive/tensor_ir/index.html">TensorIR</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../deep_dive/relax/index.html">Relax</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../../reference/api/python/index.html">Python API</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../reference/api/links.html">Other APIs</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Legacy</span></p>
-<ul class="current">
-<li class="toctree-l1"><a class="reference internal" href="../../tutorial/index.html">User Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../how_to/legacy_index.html">How To Guides</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../dev/tutorial/index.html">Developer Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../dev/how_to/how_to.html">Developer How-To Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../reference/langref/index.html">Language Reference</a></li>
-<li class="toctree-l1 current"><a class="reference internal" href="../index.html">Design and Architecture</a><ul class="current">
-<li class="toctree-l2"><a class="reference internal" href="../index.html#example-compilation-flow">Example Compilation Flow</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#logical-architecture-components">Logical Architecture Components</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#tvm-support">tvm/support</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#tvm-runtime">tvm/runtime</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#tvm-node">tvm/node</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#tvm-ir">tvm/ir</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#tvm-target">tvm/target</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#tvm-tir">tvm/tir</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#tvm-arith">tvm/arith</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#tvm-te">tvm/te</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#tvm-topi">tvm/topi</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#tvm-relay">tvm/relay</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#tvm-autotvm">tvm/autotvm</a></li>
-<li class="toctree-l2 current"><a class="reference internal" href="../index.html#frontends">Frontends</a><ul class="current">
-<li class="toctree-l3 current"><a class="current reference internal" href="#">TensorFlow Frontend</a><ul>
-<li class="toctree-l4"><a class="reference internal" href="#preparing-a-model-for-inference">Preparing a Model for Inference</a></li>
-<li class="toctree-l4"><a class="reference internal" href="#import-the-model">Import the Model</a></li>
-<li class="toctree-l4"><a class="reference internal" href="#best-practices">Best Practices</a></li>
-<li class="toctree-l4"><a class="reference internal" href="#supported-ops">Supported Ops</a></li>
-</ul>
-</li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="../index.html#microtvm">microTVM</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="../../topic/microtvm/index.html">microTVM: TVM on bare-metal</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../topic/vta/index.html">VTA: Versatile Tensor Accelerator</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">About</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../../contribute/index.html">Contributor Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../reference/publications.html">Publications</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../reference/security.html">Security Guide</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Index</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../../genindex.html">Index</a></li>
-</ul>
-
-
-
- </div>
-
- </div>
- </nav>
-
- <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
-
- <nav class="wy-nav-top" aria-label="top navigation" data-toggle="wy-nav-top">
-
- <div class="togglemenu">
-
- </div>
- <div class="nav-content">
- <!-- tvm -->
- Table of Contents
- </div>
-
- </nav>
-
-
- <div class="wy-nav-content">
-
- <div class="rst-content">
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-<div role="navigation" aria-label="breadcrumbs navigation">
-
- <ul class="wy-breadcrumbs">
-
- <li><a href="../../index.html">Docs</a> <span class="br-arrow">></span></li>
-
- <li><a href="../index.html">Design and Architecture</a> <span class="br-arrow">></span></li>
-
- <li>TensorFlow Frontend</li>
-
-
-
-
-
-
-
-
- <li class="wy-breadcrumbs-aside">
-
-
-
- <a href="https://github.com/apache/tvm/edit/main/docs/arch/frontend/tensorflow.rst" class="fa fa-github"> Edit on GitHub</a>
-
-
-
- </li>
-
- </ul>
-
-
- <hr/>
-</div>
- <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
- <div itemprop="articleBody">
-
- <div class="section" id="tensorflow-frontend">
-<h1>TensorFlow Frontend<a class="headerlink" href="#tensorflow-frontend" title="Permalink to this headline">¶</a></h1>
-<p>The TensorFlow frontend helps in importing TensorFlow models into TVM.</p>
-<p>Supported versions:</p>
-<ul class="simple">
-<li><p>1.12 and below</p></li>
-</ul>
-<p>Tested models:</p>
-<ul class="simple">
-<li><p>Inception (V1/V2/V3/V4)</p></li>
-<li><p>Resnet (All)</p></li>
-<li><p>Mobilenet (V1/V2 All)</p></li>
-<li><p>Vgg (16/19)</p></li>
-<li><p>BERT (Base/3-layer)</p></li>
-</ul>
-<div class="section" id="preparing-a-model-for-inference">
-<h2>Preparing a Model for Inference<a class="headerlink" href="#preparing-a-model-for-inference" title="Permalink to this headline">¶</a></h2>
-<div class="section" id="remove-unneeded-nodes">
-<h3>Remove Unneeded Nodes<a class="headerlink" href="#remove-unneeded-nodes" title="Permalink to this headline">¶</a></h3>
-<p>The export process will remove many nodes that are not needed for inference, but unfortunately will leave some remaining. The nodes that should be manually removed are:</p>
-<ul class="simple">
-<li><p>Dropout, including <a class="reference external" href="https://www.tensorflow.org/api_docs/python/tf/nn/dropout">Dropout</a> and <a class="reference external" href="https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/nn/rnn_cell/DropoutWrapper?hl=hr">DropoutWrapper</a></p></li>
-<li><p><a class="reference external" href="https://www.tensorflow.org/api_docs/python/tf/debugging/Assert">Assert</a></p></li>
-</ul>
-</div>
-<div class="section" id="convert-none-dimensions-to-constants">
-<h3>Convert None Dimensions to Constants<a class="headerlink" href="#convert-none-dimensions-to-constants" title="Permalink to this headline">¶</a></h3>
-<p>TVM has minimal support for dynamic tensor shapes. Dimensions that are <code class="docutils literal notranslate"><span class="pre">None</span></code> should be replaced with constants. For example, a model may accept an input with shape <code class="docutils literal notranslate"><span class="pre">(None,20)</span></code>. This should be converted to a shape like <code class="docutils literal notranslate"><span class="pre">(1,20)</span></code>. The model should be modified accordingly [...]
-</div>
-<div class="section" id="export">
-<h3>Export<a class="headerlink" href="#export" title="Permalink to this headline">¶</a></h3>
-<p>TensorFlow frontend expects a frozen protobuf (.pb) or saved model as input. It currently does not support checkpoint (.ckpt). The graphdef needed by the TensorFlow frontend can be extracted from the active session, or by using the <a class="reference external" href="https://github.com/apache/tvm/blob/main/python/tvm/relay/frontend/tensorflow_parser.py">TFParser</a> helper class.</p>
-<p>The model should be exported with a number of transformations to prepare the model for inference. It is also important to set <code class="docutils literal notranslate"><span class="pre">`add_shapes=True`</span></code>, as this will embed the output shapes of each node into the graph. Here is one function to export a model as a protobuf given a session:</p>
-<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">tensorflow</span> <span class="k">as</span> <span class="nn">tf</span>
-<span class="kn">from</span> <span class="nn">tensorflow.tools.graph_transforms</span> <span class="kn">import</span> <span class="n">TransformGraph</span>
-
-<span class="k">def</span> <span class="nf">export_pb</span><span class="p">(</span><span class="n">session</span><span class="p">):</span>
- <span class="k">with</span> <span class="n">tf</span><span class="o">.</span><span class="n">gfile</span><span class="o">.</span><span class="n">GFile</span><span class="p">(</span><span class="s2">"myexportedmodel.pb"</span><span class="p">,</span> <span class="s2">"wb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
- <span class="n">inputs</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"myinput1"</span><span class="p">,</span> <span class="s2">"myinput2"</span><span class="p">]</span> <span class="c1"># replace with your input names</span>
- <span class="n">outputs</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"myoutput1"</span><span class="p">]</span> <span class="c1"># replace with your output names</span>
- <span class="n">graph_def</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">graph</span><span class="o">.</span><span class="n">as_graph_def</span><span class="p">(</span><span class="n">add_shapes</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
- <span class="n">graph_def</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">graph</span><span class="o">.</span><span class="n">util</span><span class="o">.</span><span class="n">convert_variables_to_constants</span><span class="p">(</span><span class="n">session</span><span class="p">,</span> <span class="n">graph_def</span><span class="p">,</span> <span class="n">outputs</span><span class="p">)</span>
- <span class="n">graph_def</span> <span class="o">=</span> <span class="n">TransformGraph</span><span class="p">(</span>
- <span class="n">graph_def</span><span class="p">,</span>
- <span class="n">inputs</span><span class="p">,</span>
- <span class="n">outputs</span><span class="p">,</span>
- <span class="p">[</span>
- <span class="s2">"remove_nodes(op=Identity, op=CheckNumerics, op=StopGradient)"</span><span class="p">,</span>
- <span class="s2">"sort_by_execution_order"</span><span class="p">,</span> <span class="c1"># sort by execution order after each transform to ensure correct node ordering</span>
- <span class="s2">"remove_attribute(attribute_name=_XlaSeparateCompiledGradients)"</span><span class="p">,</span>
- <span class="s2">"remove_attribute(attribute_name=_XlaCompile)"</span><span class="p">,</span>
- <span class="s2">"remove_attribute(attribute_name=_XlaScope)"</span><span class="p">,</span>
- <span class="s2">"sort_by_execution_order"</span><span class="p">,</span>
- <span class="s2">"remove_device"</span><span class="p">,</span>
- <span class="s2">"sort_by_execution_order"</span><span class="p">,</span>
- <span class="s2">"fold_batch_norms"</span><span class="p">,</span>
- <span class="s2">"sort_by_execution_order"</span><span class="p">,</span>
- <span class="s2">"fold_old_batch_norms"</span><span class="p">,</span>
- <span class="s2">"sort_by_execution_order"</span>
- <span class="p">]</span>
- <span class="p">)</span>
- <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">graph_def</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>
-</pre></div>
-</div>
-<p>Another method is to <a class="reference external" href="https://github.com/tensorflow/models/tree/master/research/slim#exporting-the-inference-graph">export and freeze the graph</a>.</p>
-</div>
-</div>
-<div class="section" id="import-the-model">
-<h2>Import the Model<a class="headerlink" href="#import-the-model" title="Permalink to this headline">¶</a></h2>
-<div class="section" id="explicit-shape">
-<h3>Explicit Shape:<a class="headerlink" href="#explicit-shape" title="Permalink to this headline">¶</a></h3>
-<p>To ensure shapes can be known throughout the entire graph, pass the <code class="docutils literal notranslate"><span class="pre">`shape`</span></code> argument to <code class="docutils literal notranslate"><span class="pre">`from_tensorflow`</span></code>. This dictionary maps input names to input shapes. Please refer to these <a class="reference external" href="https://github.com/apache/tvm/blob/main/tests/python/frontend/tensorflow/test_forward.py#L36">test cases</a> for examples.</p>
-</div>
-<div class="section" id="data-layout">
-<h3>Data Layout<a class="headerlink" href="#data-layout" title="Permalink to this headline">¶</a></h3>
-<p>Most TensorFlow models are released with NHWC layout. NCHW layout often provides better performance, especially on GPU. The TensorFlow frontend can automatically convert the model’s data layout by passing the argument <code class="docutils literal notranslate"><span class="pre">`layout='NCHW'`</span></code> to <code class="docutils literal notranslate"><span class="pre">`from_tensorflow`</span></code>.</p>
-</div>
-</div>
-<div class="section" id="best-practices">
-<h2>Best Practices<a class="headerlink" href="#best-practices" title="Permalink to this headline">¶</a></h2>
-<ul class="simple">
-<li><p>Use static tensor shapes instead of dynamic shapes (remove <code class="docutils literal notranslate"><span class="pre">`None`</span></code> dimensions).</p></li>
-<li><p>Use static RNN instead of dynamic RNN, as <code class="docutils literal notranslate"><span class="pre">`TensorArray`</span></code> isn’t supported yet.</p></li>
-</ul>
-</div>
-<div class="section" id="supported-ops">
-<h2>Supported Ops<a class="headerlink" href="#supported-ops" title="Permalink to this headline">¶</a></h2>
-<ul class="simple">
-<li><p>Abs</p></li>
-<li><p>Add</p></li>
-<li><p>AddN</p></li>
-<li><p>All</p></li>
-<li><p>Any</p></li>
-<li><p>ArgMax</p></li>
-<li><p>ArgMin</p></li>
-<li><p>AvgPool</p></li>
-<li><p>BatchMatMul</p></li>
-<li><p>BatchMatMulV2</p></li>
-<li><p>BatchNormWithGlobalNormalization</p></li>
-<li><p>BatchToSpaceND</p></li>
-<li><p>BiasAdd</p></li>
-<li><p>BroadcastTo</p></li>
-<li><p>Cast</p></li>
-<li><p>Ceil</p></li>
-<li><p>CheckNumerics</p></li>
-<li><p>ClipByValue</p></li>
-<li><p>Concat</p></li>
-<li><p>ConcatV2</p></li>
-<li><p>Conv2D</p></li>
-<li><p>Cos</p></li>
-<li><p>Tan</p></li>
-<li><p>CropAndResize</p></li>
-<li><p>DecodeJpeg</p></li>
-<li><p>DepthwiseConv2dNative</p></li>
-<li><p>DepthToSpace</p></li>
-<li><p>Dilation2D</p></li>
-<li><p>Equal</p></li>
-<li><p>Elu</p></li>
-<li><p>Enter</p></li>
-<li><p>Erf</p></li>
-<li><p>Exit</p></li>
-<li><p>Exp</p></li>
-<li><p>ExpandDims</p></li>
-<li><p>Fill</p></li>
-<li><p>Floor</p></li>
-<li><p>FloorDiv</p></li>
-<li><p>FloorMod</p></li>
-<li><p>FusedBatchNorm</p></li>
-<li><p>FusedBatchNormV2</p></li>
-<li><p>Gather</p></li>
-<li><p>GatherNd</p></li>
-<li><p>GatherV2</p></li>
-<li><p>Greater</p></li>
-<li><p>GreaterEqual</p></li>
-<li><p>Identity</p></li>
-<li><p>IsFinite</p></li>
-<li><p>IsInf</p></li>
-<li><p>IsNan</p></li>
-<li><p>LeakyRelu</p></li>
-<li><p>LeftShift</p></li>
-<li><p>Less</p></li>
-<li><p>LessEqual</p></li>
-<li><p>Log</p></li>
-<li><p>Log1p</p></li>
-<li><p>LoopCond</p></li>
-<li><p>LogicalAnd</p></li>
-<li><p>LogicalOr</p></li>
-<li><p>LogicalNot</p></li>
-<li><p>LogSoftmax</p></li>
-<li><p>LRN</p></li>
-<li><p>LSTMBlockCell</p></li>
-<li><p>MatMul</p></li>
-<li><p>Max</p></li>
-<li><p>MaxPool</p></li>
-<li><p>Maximum</p></li>
-<li><p>Mean</p></li>
-<li><p>Merge</p></li>
-<li><p>Min</p></li>
-<li><p>Minimum</p></li>
-<li><p>MirrorPad</p></li>
-<li><p>Mod</p></li>
-<li><p>Mul</p></li>
-<li><p>Neg</p></li>
-<li><p>NextIteration</p></li>
-<li><p>NotEqual</p></li>
-<li><p>OneHot</p></li>
-<li><p>Pack</p></li>
-<li><p>Pad</p></li>
-<li><p>PadV2</p></li>
-<li><p>Pow</p></li>
-<li><p>Prod</p></li>
-<li><p>Range</p></li>
-<li><p>Rank</p></li>
-<li><p>RealDiv</p></li>
-<li><p>Relu</p></li>
-<li><p>Relu6</p></li>
-<li><p>Reshape</p></li>
-<li><p>ResizeBilinear</p></li>
-<li><p>ResizeBicubic</p></li>
-<li><p>ResizeNearestNeighbor</p></li>
-<li><p>ReverseV2</p></li>
-<li><p>RightShift</p></li>
-<li><p>Round</p></li>
-<li><p>Rsqrt</p></li>
-<li><p>Select</p></li>
-<li><p>Selu</p></li>
-<li><p>Shape</p></li>
-<li><p>Sigmoid</p></li>
-<li><p>Sign</p></li>
-<li><p>Sin</p></li>
-<li><p>Size</p></li>
-<li><p>Slice</p></li>
-<li><p>Softmax</p></li>
-<li><p>Softplus</p></li>
-<li><p>SpaceToBatchND</p></li>
-<li><p>SpaceToDepth,</p></li>
-<li><p>Split</p></li>
-<li><p>SplitV</p></li>
-<li><p>Sqrt</p></li>
-<li><p>Square</p></li>
-<li><p>SquareDifference</p></li>
-<li><p>Squeeze</p></li>
-<li><p>StridedSlice</p></li>
-<li><p>Sub</p></li>
-<li><p>Sum</p></li>
-<li><p>Switch</p></li>
-<li><p>Tanh</p></li>
-<li><p>TensorArrayV3</p></li>
-<li><p>TensorArrayScatterV3</p></li>
-<li><p>TensorArrayGatherV3</p></li>
-<li><p>TensorArraySizeV3</p></li>
-<li><p>TensorArrayWriteV3</p></li>
-<li><p>TensorArrayReadV3</p></li>
-<li><p>TensorArraySplitV3</p></li>
-<li><p>TensorArrayConcatV3</p></li>
-<li><p>Tile</p></li>
-<li><p>TopKV2</p></li>
-<li><p>Transpose</p></li>
-<li><p>TruncateMod</p></li>
-<li><p>Unpack</p></li>
-<li><p>UnravelIndex</p></li>
-<li><p>Where</p></li>
-<li><p>ZerosLike</p></li>
-</ul>
-</div>
-</div>
-
-
- </div>
-
- </div>
-
-
-<footer>
-
- <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
-
- <a href="../microtvm_design.html" class="btn btn-neutral float-right" title="microTVM Design Document" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
-
-
- <a href="../benchmark.html" class="btn btn-neutral float-left" title="Benchmark Performance Log Format" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
-
- </div>
-
-<div id="button" class="backtop"><img src="../../_static/img/right.svg" alt="backtop"/> </div>
-<section class="footerSec">
- <div class="footerHeader">
- <div class="d-flex align-md-items-center justify-content-between flex-column flex-md-row">
- <div class="copywrite d-flex align-items-center">
- <h5 id="copy-right-info">© 2024 Apache Software Foundation | All rights reserved</h5>
- </div>
- </div>
-
- </div>
-
- <div>
- <div class="footernote">Copyright © 2024 The Apache Software Foundation. Apache TVM, Apache, the Apache feather, and the Apache TVM project logo are either trademarks or registered trademarks of the Apache Software Foundation.</div>
- </div>
-
-</section>
-</footer>
- </div>
- </div>
-
- </section>
-
- </div>
-
-
- <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script>
- <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script>
-
- </body>
- <script type="text/javascript">
- jQuery(function () {
- SphinxRtdTheme.Navigation.enable(true);
- });
- </script>
-
-
-
-
- <!-- Theme Analytics -->
- <script>
- (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
- (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
- m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
- })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
-
- ga('create', 'UA-75982049-2', 'auto');
- ga('send', 'pageview');
- </script>
-
-
-
-
-</body>
-</html>
\ No newline at end of file
diff --git a/docs/arch/hybrid_script.html b/docs/arch/hybrid_script.html
deleted file mode 100644
index cd0813d84b..0000000000
--- a/docs/arch/hybrid_script.html
+++ /dev/null
@@ -1,551 +0,0 @@
-
-
-
-
-
-
-<!DOCTYPE html>
-<html class="writer-html5" lang="en" >
-<head>
- <meta charset="utf-8">
-
- <meta name="viewport" content="width=device-width, initial-scale=1.0">
-
- <title>Hybrid Frontend Developer Guide — tvm 0.18.dev0 documentation</title>
-
-
-
- <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
- <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
- <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
- <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-binder.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-dataframe.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-rendered-html.css" type="text/css" />
- <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
- <link rel="stylesheet" href="../_static/css/tlcpack_theme.css" type="text/css" />
-
-
-
- <link rel="shortcut icon" href="../_static/tvm-logo-square.png"/>
-
-
-
-
-
-
-
- <script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
- <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
- <script src="../_static/jquery.js"></script>
- <script src="../_static/underscore.js"></script>
- <script src="../_static/doctools.js"></script>
-
- <script type="text/javascript" src="../_static/js/theme.js"></script>
-
-
- <script type="text/javascript" src="../_static/js/tlcpack_theme.js"></script>
- <link rel="index" title="Index" href="../genindex.html" />
- <link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Introduction to Relay IR" href="relay_intro.html" />
- <link rel="prev" title="InferBound Pass" href="inferbound.html" />
-</head>
-
-<body class="wy-body-for-nav">
-
-
- <div class="wy-grid-for-nav">
-
-
-<header class="header">
- <div class="innercontainer">
- <div class="headerInner d-flex justify-content-between align-items-center">
- <div class="headerLogo">
- <a href="https://tvm.apache.org/"><img src=https://tvm.apache.org/assets/images/logo.svg alt="logo"></a>
- </div>
-
- <div id="headMenu" class="headerNav">
- <button type="button" id="closeHeadMenu" class="navCloseBtn"><img src="../_static/img/close-icon.svg" alt="Close"></button>
- <ul class="nav">
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/community>Community</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/download>Download</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/blog>Blog</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/docs>Docs</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvmconf.org>Conference</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://github.com/apache/tvm/>Github</a>
- </li>
- </ul>
- <div class="responsivetlcdropdown">
- <button type="button" class="btn-link">
- ASF
- </button>
- <ul>
- <li>
- <a href=https://apache.org/>Apache Homepage</a>
- </li>
- <li>
- <a href=https://www.apache.org/licenses/>License</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/sponsorship.html>Sponsorship</a>
- </li>
- <li>
- <a href=https://tvm.apache.org/docs/reference/security.html>Security</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/thanks.html>Thanks</a>
- </li>
- <li>
- <a href=https://www.apache.org/events/current-event>Events</a>
- </li>
- </ul>
- </div>
- </div>
- <div class="responsiveMenuIcon">
- <button type="button" id="menuBtn" class="btn-menu"><img src="../_static/img/menu-icon.svg" alt="Menu Icon"></button>
- </div>
-
- <div class="tlcDropdown">
- <div class="dropdown">
- <button type="button" class="btn-link dropdown-toggle" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
- ASF
- </button>
- <div class="dropdown-menu dropdown-menu-right">
- <ul>
- <li>
- <a href=https://apache.org/>Apache Homepage</a>
- </li>
- <li>
- <a href=https://www.apache.org/licenses/>License</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/sponsorship.html>Sponsorship</a>
- </li>
- <li>
- <a href=https://tvm.apache.org/docs/reference/security.html>Security</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/thanks.html>Thanks</a>
- </li>
- <li>
- <a href=https://www.apache.org/events/current-event>Events</a>
- </li>
- </ul>
- </div>
- </div>
- </div>
- </div>
- </div>
- </header>
-
- <nav data-toggle="wy-nav-shift" class="wy-nav-side fixed">
- <div class="wy-side-scroll">
- <div class="wy-side-nav-search" >
-
-
-
- <a href="../index.html">
-
-
-
-
- <img src="../_static/tvm-logo-small.png" class="logo" alt="Logo"/>
-
- </a>
-
-
-
-
- <input type="checkbox" class="version-toggle-box" hidden id="version-toggle">
- <label for="version-toggle" class="version-toggle-label">
- <div tabindex="0" class="version version-selector version-selector-show">
- 0.18.dev0 <span class="chevron versions-hidden"><svg fill="none" height="24" viewBox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg"><path d="m8 4 8 8-8 8" stroke="#000" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"/></svg></span><span class="chevron versions-shown"><svg fill="none" height="24" viewBox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg"><path d="m4 8 8 8 8-8" stroke="#000" stroke-linecap="round" stroke-linejoin="round [...]
- </div>
- </label>
- <div class="version-details wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
- <p class="caption" role="heading"><span class="caption-text">Versions</span></p>
- <ol style="text-align: left">
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="/">0.18.dev0 (main)</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.8.0/">v0.8.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.9.0/">v0.9.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.10.0/">v0.10.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.11.0/">v0.11.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.12.0/">v0.12.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.13.0/">v0.13.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.14.0/">v0.14.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.15.0/">v0.15.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.16.0/">v0.16.0</a></div></li>
-
- </ol>
- </div>
-
-
-
-
-<div role="search">
- <form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
- <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
- </form>
-</div>
-
-
- </div>
-
-
- <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
-
-
-
-
-
-
- <p class="caption" role="heading"><span class="caption-text">Getting Started</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/overview.html">Overview</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../install/index.html">Installing TVM</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/tutorials/quick_start.html">Quick Start</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/tutorials/ir_module.html">IRModule</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">How To</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/e2e_opt_model.html">End-to-End Optimize Model</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/customize_opt.html">Customize Optimization</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/optimize_llm.html">Optimize Large Language Model</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/cross_compilation_and_rpc.html">Cross Compilation and RPC</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/dev/index.html">Development Guides</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Deep Dive</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/tensor_ir/index.html">TensorIR</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/relax/index.html">Relax</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/python/index.html">Python API</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/links.html">Other APIs</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Legacy</span></p>
-<ul class="current">
-<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">User Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/legacy_index.html">How To Guides</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/tutorial/index.html">Developer Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/how_to/how_to.html">Developer How-To Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/langref/index.html">Language Reference</a></li>
-<li class="toctree-l1 current"><a class="reference internal" href="index.html">Design and Architecture</a><ul class="current">
-<li class="toctree-l2"><a class="reference internal" href="index.html#example-compilation-flow">Example Compilation Flow</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#logical-architecture-components">Logical Architecture Components</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-support">tvm/support</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-runtime">tvm/runtime</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-node">tvm/node</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-ir">tvm/ir</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-target">tvm/target</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-tir">tvm/tir</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-arith">tvm/arith</a></li>
-<li class="toctree-l2 current"><a class="reference internal" href="index.html#tvm-te">tvm/te</a><ul class="current">
-<li class="toctree-l3"><a class="reference internal" href="inferbound.html">InferBound Pass</a></li>
-<li class="toctree-l3 current"><a class="current reference internal" href="#">Hybrid Frontend Developer Guide</a><ul>
-<li class="toctree-l4"><a class="reference internal" href="#features">Features</a></li>
-</ul>
-</li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-topi">tvm/topi</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-relay">tvm/relay</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-autotvm">tvm/autotvm</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#frontends">Frontends</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#microtvm">microTVM</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="../topic/microtvm/index.html">microTVM: TVM on bare-metal</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../topic/vta/index.html">VTA: Versatile Tensor Accelerator</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">About</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../contribute/index.html">Contributor Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/publications.html">Publications</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/security.html">Security Guide</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Index</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../genindex.html">Index</a></li>
-</ul>
-
-
-
- </div>
-
- </div>
- </nav>
-
- <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
-
- <nav class="wy-nav-top" aria-label="top navigation" data-toggle="wy-nav-top">
-
- <div class="togglemenu">
-
- </div>
- <div class="nav-content">
- <!-- tvm -->
- Table of Contents
- </div>
-
- </nav>
-
-
- <div class="wy-nav-content">
-
- <div class="rst-content">
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-<div role="navigation" aria-label="breadcrumbs navigation">
-
- <ul class="wy-breadcrumbs">
-
- <li><a href="../index.html">Docs</a> <span class="br-arrow">></span></li>
-
- <li><a href="index.html">Design and Architecture</a> <span class="br-arrow">></span></li>
-
- <li>Hybrid Frontend Developer Guide</li>
-
-
-
-
-
-
-
-
- <li class="wy-breadcrumbs-aside">
-
-
-
- <a href="https://github.com/apache/tvm/edit/main/docs/arch/hybrid_script.rst" class="fa fa-github"> Edit on GitHub</a>
-
-
-
- </li>
-
- </ul>
-
-
- <hr/>
-</div>
- <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
- <div itemprop="articleBody">
-
- <div class="section" id="hybrid-frontend-developer-guide">
-<h1>Hybrid Frontend Developer Guide<a class="headerlink" href="#hybrid-frontend-developer-guide" title="Permalink to this headline">¶</a></h1>
-<p>If you are a developer:</p>
-<p>1. who is trying writing some preliminary patterns that have not been supported by TVM yet,
-maybe <a class="reference internal" href="../reference/langref/hybrid_script.html#hybrid-langref-label"><span class="std std-ref">Hybrid Frontend Language Reference</span></a> is a better place for you.</p>
-<ol class="arabic simple" start="2">
-<li><p>who wants to know the implementation details of this module, you are right here!</p></li>
-</ol>
-<div class="section" id="features">
-<h2>Features<a class="headerlink" href="#features" title="Permalink to this headline">¶</a></h2>
-<div class="section" id="software-emulation">
-<h3>Software Emulation<a class="headerlink" href="#software-emulation" title="Permalink to this headline">¶</a></h3>
-<p>In software emulation, the most interesting thing is the decorator <code class="docutils literal notranslate"><span class="pre">tvm.te.hybrid.script</span></code>.
-This decorator helps 2 things:</p>
-<ol class="arabic simple">
-<li><p>Importing runtime variables</p></li>
-<li><p>Overloading the function according to the arguments passed</p></li>
-</ol>
-<p>Correct me if I am wrong: I believe that how 1. is implemented is dangerous, but I have no
-choice. What I did is to add those names into python dict <code class="docutils literal notranslate"><span class="pre">func.__global__</span></code> and after
-the call to <code class="docutils literal notranslate"><span class="pre">func</span></code> is done, those names will be cleaned up.</p>
-<p>Overload is simple: the decorator checks the arguments’ types and determines which function
-should be actually called.</p>
-</div>
-<div class="section" id="backend-compilation">
-<h3>Backend Compilation<a class="headerlink" href="#backend-compilation" title="Permalink to this headline">¶</a></h3>
-<p>Compilation is a large module, you can see <code class="docutils literal notranslate"><span class="pre">python/tvm/te/hybrid/</span></code> for more
-details. The first stage determines the usage, or more accurately the
-declaration of each variable and the second stage does the actual IR
-generation.</p>
-</div>
-<div class="section" id="attributes">
-<h3>Attributes<a class="headerlink" href="#attributes" title="Permalink to this headline">¶</a></h3>
-<p>So far, ONLY tensors’ <cite>shape</cite> attribute is supported. You can see <code class="docutils literal notranslate"><span class="pre">visit_Subscript</span></code>
-in <code class="docutils literal notranslate"><span class="pre">python/tvm/te/hybrid/parser.py</span></code> for more details. This is a hacky solution, I just
-check the attributes when subscript.</p>
-</div>
-<div class="section" id="loops">
-<h3>Loops<a class="headerlink" href="#loops" title="Permalink to this headline">¶</a></h3>
-<p>In HalideIR, loops have in total 4 types: <code class="docutils literal notranslate"><span class="pre">serial</span></code>, <code class="docutils literal notranslate"><span class="pre">unrolled</span></code>, <code class="docutils literal notranslate"><span class="pre">parallel</span></code>, and <code class="docutils literal notranslate"><span class="pre">vectorized</span></code>.</p>
-<div class="admonition note">
-<p class="admonition-title">Note</p>
-<p>Unlike what that is in HalideIR, in <code class="docutils literal notranslate"><span class="pre">loop_type(a,</span> <span class="pre">b)</span></code>, <code class="docutils literal notranslate"><span class="pre">a</span></code> is the starting point and <code class="docutils literal notranslate"><span class="pre">b</span></code>
-is the trip count of iterations. Here <code class="docutils literal notranslate"><span class="pre">loop_type(a,</span> <span class="pre">b)</span></code> indicates <code class="docutils literal notranslate"><span class="pre">[a,</span> <span class="pre">b)</span></code>. Thus, when lowering it
-to HalideIR, we need to do <code class="docutils literal notranslate"><span class="pre">start,</span> <span class="pre">extent</span> <span class="pre">=</span> <span class="pre">a,</span> <span class="pre">b</span> <span class="pre">-</span> <span class="pre">a</span></code></p>
-</div>
-<div class="admonition note">
-<p class="admonition-title">Note</p>
-<p>In HalideIR those are enums, they are in passive form.
-Here we use active form to annotate loops, because they are ready to run.</p>
-</div>
-</div>
-<div class="section" id="variables">
-<h3>Variables<a class="headerlink" href="#variables" title="Permalink to this headline">¶</a></h3>
-<p>Because there is no variables in <code class="docutils literal notranslate"><span class="pre">HalideIR</span></code>, all the mutable variables will be lowered to an array with size 1.
-It takes the first store of a variable as its declaration.</p>
-</div>
-<div class="section" id="math-intrinsics">
-<h3>Math Intrinsics<a class="headerlink" href="#math-intrinsics" title="Permalink to this headline">¶</a></h3>
-<p>So far, these math intrinsics, <code class="docutils literal notranslate"><span class="pre">log</span></code>, <code class="docutils literal notranslate"><span class="pre">exp</span></code>, <code class="docutils literal notranslate"><span class="pre">sigmoid</span></code>, <code class="docutils literal notranslate"><span class="pre">tanh</span></code>, <code class="docutils literal notranslate"><span class="pre">power</span></code>, and <code class="docutils literal notranslate"><spa [...]
-Math intrinsics will be imported by the decorator. Most of the intrinsics are borrowed by library implementation
-except <code class="docutils literal notranslate"><span class="pre">popcount</span></code> and <code class="docutils literal notranslate"><span class="pre">sigmoid</span></code>. I implemented them manually.</p>
-</div>
-<div class="section" id="casting">
-<h3>Casting<a class="headerlink" href="#casting" title="Permalink to this headline">¶</a></h3>
-<p>You can cast values by using the keywords <code class="docutils literal notranslate"><span class="pre">uint8</span></code>, <code class="docutils literal notranslate"><span class="pre">uint16</span></code> <code class="docutils literal notranslate"><span class="pre">uint32</span></code>, <code class="docutils literal notranslate"><span class="pre">uint64</span></code>, <code class="docutils literal notranslate"><span class="pre">int8</span></code>, <code class="docutils literal notran [...]
-<code class="docutils literal notranslate"><span class="pre">float16</span></code>, <code class="docutils literal notranslate"><span class="pre">float32</span></code>, <code class="docutils literal notranslate"><span class="pre">float64</span></code>.</p>
-</div>
-</div>
-</div>
-
-
- </div>
-
- </div>
-
-
-<footer>
-
- <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
-
- <a href="relay_intro.html" class="btn btn-neutral float-right" title="Introduction to Relay IR" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
-
-
- <a href="inferbound.html" class="btn btn-neutral float-left" title="InferBound Pass" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
-
- </div>
-
-<div id="button" class="backtop"><img src="../_static/img/right.svg" alt="backtop"/> </div>
-<section class="footerSec">
- <div class="footerHeader">
- <div class="d-flex align-md-items-center justify-content-between flex-column flex-md-row">
- <div class="copywrite d-flex align-items-center">
- <h5 id="copy-right-info">© 2024 Apache Software Foundation | All rights reserved</h5>
- </div>
- </div>
-
- </div>
-
- <div>
- <div class="footernote">Copyright © 2024 The Apache Software Foundation. Apache TVM, Apache, the Apache feather, and the Apache TVM project logo are either trademarks or registered trademarks of the Apache Software Foundation.</div>
- </div>
-
-</section>
-</footer>
- </div>
- </div>
-
- </section>
-
- </div>
-
-
- <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script>
- <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script>
-
- </body>
- <script type="text/javascript">
- jQuery(function () {
- SphinxRtdTheme.Navigation.enable(true);
- });
- </script>
-
-
-
-
- <!-- Theme Analytics -->
- <script>
- (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
- (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
- m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
- })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
-
- ga('create', 'UA-75982049-2', 'auto');
- ga('send', 'pageview');
- </script>
-
-
-
-
-</body>
-</html>
\ No newline at end of file
diff --git a/docs/arch/index.html b/docs/arch/index.html
index 2d2e864a8d..fdcdc02a37 100644
--- a/docs/arch/index.html
+++ b/docs/arch/index.html
@@ -49,7 +49,7 @@
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="TVM Runtime System" href="runtime.html" />
- <link rel="prev" title="Hybrid Frontend Language Reference" href="../reference/langref/hybrid_script.html" />
+ <link rel="prev" title="Handle TVM Errors" href="../errors.html" />
</head>
<body class="wy-body-for-nav">
@@ -269,39 +269,26 @@
<li class="toctree-l1"><a class="reference internal" href="../how_to/dev/index.html">Development Guides</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Deep Dive</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/tensor_ir/index.html">TensorIR</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/relax/index.html">Relax</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/python/index.html">Python API</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/links.html">Other APIs</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Legacy</span></p>
<ul class="current">
-<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">User Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/legacy_index.html">How To Guides</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/tutorial/index.html">Developer Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/how_to/how_to.html">Developer How-To Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/langref/index.html">Language Reference</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Design and Architecture</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="#example-compilation-flow">Example Compilation Flow</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#overall-flow">Overall Flow</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#key-data-structures">Key data structures</a></li>
-<li class="toctree-l3"><a class="reference internal" href="#transformations">Transformations</a></li>
-<li class="toctree-l3"><a class="reference internal" href="#search-space-and-learning-based-transformations">Search-space and Learning-based Transformations</a></li>
+<li class="toctree-l3"><a class="reference internal" href="#transformations">Transformations</a><ul>
+<li class="toctree-l4"><a class="reference internal" href="#relax-transformations">relax transformations</a></li>
+<li class="toctree-l4"><a class="reference internal" href="#tir-transformations">tir transformations</a></li>
+<li class="toctree-l4"><a class="reference internal" href="#cross-level-transformations">cross-level transformations</a></li>
+</ul>
+</li>
<li class="toctree-l3"><a class="reference internal" href="#target-translation">Target Translation</a></li>
<li class="toctree-l3"><a class="reference internal" href="#runtime-execution">Runtime Execution</a></li>
<li class="toctree-l3"><a class="reference internal" href="#summary-and-discussions">Summary and Discussions</a></li>
</ul>
</li>
-<li class="toctree-l2"><a class="reference internal" href="#logical-architecture-components">Logical Architecture Components</a></li>
<li class="toctree-l2"><a class="reference internal" href="#tvm-support">tvm/support</a></li>
<li class="toctree-l2"><a class="reference internal" href="#tvm-runtime">tvm/runtime</a><ul>
<li class="toctree-l3"><a class="reference internal" href="runtime.html">TVM Runtime System</a></li>
<li class="toctree-l3"><a class="reference internal" href="runtime.html#runtime-specific-information">Runtime-Specific Information</a></li>
<li class="toctree-l3"><a class="reference internal" href="debugger.html">Debugger</a></li>
-<li class="toctree-l3"><a class="reference internal" href="virtual_machine.html">Putting the VM in TVM: The Relay Virtual Machine</a></li>
<li class="toctree-l3"><a class="reference internal" href="introduction_to_module_serialization.html">Introduction to Module Serialization</a></li>
<li class="toctree-l3"><a class="reference internal" href="device_target_interactions.html">Device/Target Interactions</a></li>
</ul>
@@ -315,36 +302,29 @@
<li class="toctree-l3"><a class="reference internal" href="device_target_interactions.html">Device/Target Interactions</a></li>
</ul>
</li>
+<li class="toctree-l2"><a class="reference internal" href="#tvm-relax">tvm/relax</a></li>
<li class="toctree-l2"><a class="reference internal" href="#tvm-tir">tvm/tir</a></li>
<li class="toctree-l2"><a class="reference internal" href="#tvm-arith">tvm/arith</a></li>
-<li class="toctree-l2"><a class="reference internal" href="#tvm-te">tvm/te</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="inferbound.html">InferBound Pass</a></li>
-<li class="toctree-l3"><a class="reference internal" href="hybrid_script.html">Hybrid Frontend Developer Guide</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#tvm-te-and-tvm-topi">tvm/te and tvm/topi</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#tvm-meta-schedule">tvm/meta_schedule</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#tvm-dlight">tvm/dlight</a></li>
</ul>
</li>
-<li class="toctree-l2"><a class="reference internal" href="#tvm-topi">tvm/topi</a></li>
-<li class="toctree-l2"><a class="reference internal" href="#tvm-relay">tvm/relay</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="relay_intro.html">Introduction to Relay IR</a></li>
-<li class="toctree-l3"><a class="reference internal" href="relay_op_strategy.html">Relay Operator Strategy</a></li>
-<li class="toctree-l3"><a class="reference internal" href="convert_layout.html">Convert Layout Pass</a></li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="#tvm-autotvm">tvm/autotvm</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="benchmark.html">Benchmark Performance Log Format</a></li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="#frontends">Frontends</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="frontend/tensorflow.html">TensorFlow Frontend</a></li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="#microtvm">microTVM</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="microtvm_design.html">microTVM Design Document</a></li>
-<li class="toctree-l3"><a class="reference internal" href="microtvm_project_api.html">microTVM Project API</a></li>
-<li class="toctree-l3"><a class="reference internal" href="model_library_format.html">Model Library Format</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../deep_dive/tensor_ir/index.html">TensorIR</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../deep_dive/relax/index.html">Relax</a></li>
</ul>
-</li>
+<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../reference/api/python/index.html">Python API</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../reference/api/links.html">Other APIs</a></li>
</ul>
-</li>
+<p class="caption" role="heading"><span class="caption-text">Legacy</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">User Tutorial</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../how_to/legacy_index.html">How To Guides</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../dev/tutorial/index.html">Developer Tutorial</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../dev/how_to/how_to.html">Developer How-To Guide</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../reference/langref/index.html">Language Reference</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topic/microtvm/index.html">microTVM: TVM on bare-metal</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topic/vta/index.html">VTA: Versatile Tensor Accelerator</a></li>
</ul>
@@ -442,38 +422,33 @@
<div class="section" id="design-and-architecture">
<h1>Design and Architecture<a class="headerlink" href="#design-and-architecture" title="Permalink to this headline">¶</a></h1>
-<p>This document is intended for developers who want to understand the
-architecture of TVM and/or actively develop on the project.
+<p>This document is intended for developers who want to understand the architecture of Apache TVM and/or actively develop on the project.
This page is organized as follows:</p>
<ul class="simple">
-<li><p>The <a class="reference internal" href="#example-compilation-flow">Example Compilation Flow</a> gives an overview of the steps that TVM takes to turn a high level description of a model into a deployable module.
+<li><p>The <a class="reference internal" href="#overall-flow">Overall Flow</a> gives an overview of the steps that TVM takes to turn a high level description of a model into a deployable module.
To get started, please read this section first.</p></li>
-<li><p>The <a class="reference internal" href="#logical-architecture-components">Logical Architecture Components</a> section describes the logical components.
-The sections after are specific guides focused on each logical component, organized
-by the component’s name.</p></li>
-<li><p>The <a class="reference internal" href="device_target_interactions.html#tvm-target-specific-overview"><span class="std std-ref">Device/Target Interactions</span></a>
-page describes how TVM interacts with each supported physical device
-and code-generation target.</p></li>
-<li><p>Feel free to also check out the <a class="reference internal" href="../dev/how_to/how_to.html#dev-how-to"><span class="std std-ref">Developer How-To Guide</span></a> for useful development tips.</p></li>
+<li><p>Brief introduction to the key components of the TVM stack. Feel free to also check out the <a class="reference internal" href="../deep_dive/tensor_ir/index.html#tensor-ir-deep-dive"><span class="std std-ref">TensorIR Deep Dive</span></a>
+and <a class="reference internal" href="../deep_dive/relax/index.html#relax-deep-dive"><span class="std std-ref">Relax Deep Dive</span></a> for more details about the two major components in the TVM stack.</p></li>
</ul>
<p>This guide provides a few complementary views of the architecture.
First, we review a single end-to-end compilation flow and discuss the key data structures and the transformations.
This runtime-based view focuses on the interactions of each components when running the compiler.
Then we will review the logical modules of the codebase and their relationship. This part provides a static overarching view of the design.</p>
-<div class="section" id="example-compilation-flow">
-<h2>Example Compilation Flow<a class="headerlink" href="#example-compilation-flow" title="Permalink to this headline">¶</a></h2>
+<div class="section" id="overall-flow">
+<h2>Overall Flow<a class="headerlink" href="#overall-flow" title="Permalink to this headline">¶</a></h2>
<p>In this guide, we will study an example compilation flow in the compiler. The figure below shows the flow. At a high-level, it contains several steps:</p>
<ul class="simple">
-<li><p>Import: The frontend component ingests a model into an IRModule, which contains a collection of functions that internally represent the model.</p></li>
-<li><p>Transformation: The compiler transforms an IRModule to another functionally equivalent or approximately
+<li><p><strong>Model Creation</strong>: Create the IRModule to be optimized and compiled, which contains a collection of functions that internally represent the model.
+Users can manually construct IRModule via NNModule, TVMScript, or import a pre-trained model from from Relax frontend.</p></li>
+<li><p><strong>Transformation</strong>: The compiler transforms an IRModule to another functionally equivalent or approximately
equivalent(e.g. in the case of quantization) IRModule. Many of the transformations are target (backend) independent.
We also allow target to affect the configuration of the transformation pipeline.</p></li>
-<li><p>Target Translation: The compiler translates(codegen) the IRModule to an executable format specified by the target.
+<li><p><strong>Target Translation</strong>: The compiler translates(codegen) the IRModule to an executable format specified by the target.
The target translation result is encapsulated as a <cite>runtime.Module</cite> that can be exported, loaded, and executed on the target runtime environment.</p></li>
-<li><p>Runtime Execution: the user loads back a <cite>runtime.Module</cite> and runs the compiled functions in the supported runtime environment.</p></li>
+<li><p><strong>Runtime Execution</strong>: the user loads back a <cite>runtime.Module</cite> and runs the compiled functions in the supported runtime environment.</p></li>
</ul>
<div class="figure align-center">
-<a class="reference internal image-reference" href="https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_dyn_workflow.svg"><img alt="https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_dyn_workflow.svg" src="https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_dyn_workflow.svg" width="85%" /></a>
+<a class="reference internal image-reference" href="https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_overall_flow.svg"><img alt="https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_overall_flow.svg" src="https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_overall_flow.svg" width="80%" /></a>
</div>
<div class="section" id="key-data-structures">
<h3>Key data structures<a class="headerlink" href="#key-data-structures" title="Permalink to this headline">¶</a></h3>
@@ -483,13 +458,14 @@ components that either define a collection of key data structures or transformat
<p><strong>IRModule</strong> is the primary data structure used across the entire stack. An IRModule (intermediate representation module)
contains a collection of functions. Currently, we support two primary variants of functions.</p>
<ul class="simple">
-<li><p><strong>relay::Function</strong> is a high-level functional program representation. A relay.Function usually corresponds to an end-to-end model.
-You can view a relay.Function as a computational graph with additional support for control-flow, recursion, and complex data structures.</p></li>
+<li><p><strong>relax::Function</strong> is a high-level functional program representation. A relax.Function represents high-level graph structure,
+usually corresponds to an end-to-end model or a sub-graph of the overall model. You can view a relax.Function as a computational
+graph with additional support for control-flow, and complex data structures.</p></li>
<li><p><strong>tir::PrimFunc</strong> is a low-level program representation that contains elements including loop-nest choices, multi-dimensional load/store,
threading, and vector/tensor instructions. It is usually used to represent an operator program that executes a (possibly-fused) layer in a model.</p></li>
</ul>
-<p>During the compilation, a relay function may be lowered to multiple tir::PrimFunc functions and a top-level function that calls into
-those tir::PrimFunc functions.</p>
+<p>During the compilation and transformation, all relax operators are lowered to <code class="docutils literal notranslate"><span class="pre">tir::PrimFunc</span></code> or <code class="docutils literal notranslate"><span class="pre">TVM</span> <span class="pre">PackedFunc</span></code>, which can be executed directly
+on the target device, while the calls to relax operators are lowered to calls to low-level functions (e.g. <code class="docutils literal notranslate"><span class="pre">R.call_tir</span></code> or <code class="docutils literal notranslate"><span class="pre">R.call_dps</span></code>).</p>
</div>
<div class="section" id="transformations">
<h3>Transformations<a class="headerlink" href="#transformations" title="Permalink to this headline">¶</a></h3>
@@ -498,36 +474,37 @@ those tir::PrimFunc functions.</p>
<li><p>optimization: transform a program to an equivalent, possibly more optimized version.</p></li>
<li><p>lowering: transform a program to a lower-level representation that is closer to the target.</p></li>
</ul>
-<p><strong>relay/transform</strong> contains a collection of passes that optimize the model. The optimizations include common program
-optimizations such as constant folding and dead-code elimination, and tensor-computation specific passes such as layout
-transformation and scaling factor folding.</p>
-<p>Near the end of the relay optimization pipeline, we will run a pass(FuseOps) to break the end-to-end function(e.g. MobileNet)
-into sub-function(e.g. conv2d-relu) segments. We call these segments of functions.
-This process helps us to divide the original problem into two sub-problems:</p>
+<div class="section" id="relax-transformations">
+<h4>relax transformations<a class="headerlink" href="#relax-transformations" title="Permalink to this headline">¶</a></h4>
+<p>relax transformations contain a collection of passes that apply to relax functions. The optimizations include common graph-level
+optimizations such as constant folding and dead-code elimination for operators, and backend-specific optimizations such as library dispatch.</p>
+</div>
+<div class="section" id="tir-transformations">
+<h4>tir transformations<a class="headerlink" href="#tir-transformations" title="Permalink to this headline">¶</a></h4>
+<p>tir transformations contain a collection of passes that apply to tir functions. There are two major types of transformations:</p>
<ul class="simple">
-<li><p>Compilation and optimization for each sub-function.</p></li>
-<li><p>Overall execution structure: we need to do a sequence of calls into the generated sub-functions to execute the whole model.</p></li>
-</ul>
-<p>We use the low-level tir phase to compile and optimize each sub-functions. For specific targets, we may also directly go to the target translation
-phase and use external code generators.</p>
-<p>There are a few different ways(in relay/backend) to handle the calls into the overall execution problem. For simple models with known shapes and no control flow, we can lower to a graph executor that stores the execution structure in a graph. We also support a virtual machine backend for dynamic executions. Finally, we plan to support ahead of time compilation that compiles the high-level execution structure into the executable and generated primitive functions. All of these execution [...]
-<p><strong>tir/transform</strong> contains transformation passes for TIR level functions. Many tir passes serve the purpose of lowering. For example, there are passes to flatten multi-dimensional access to one-dimensional pointer access, to expand the intrinsics into target-specific ones, and to decorate the function entry to meet the runtime calling convention. Of course, there are also optimizations passes, such as access index simplification and dead code elimination.</p>
-<p>Many low-level optimizations can be handled in the target phase by the LLVM, CUDA C, and other target compilers. As a result, we leave low-level optimizations such as register allocation to the downstream compilers and only focus on optimizations that are not covered by them.</p>
-</div>
-<div class="section" id="search-space-and-learning-based-transformations">
-<h3>Search-space and Learning-based Transformations<a class="headerlink" href="#search-space-and-learning-based-transformations" title="Permalink to this headline">¶</a></h3>
-<p>The transformation passes we described so far are deterministic and rule-based. One design goal of the TVM stack is to support high-performance code optimizations for different hardware platforms. To do so, we will need to investigate as many optimization choices as possible, including but not limited to, multi-dimensional tensor access, loop tiling behavior, special accelerator memory hierarchy, and threading.</p>
-<p>It is hard to define a heuristic to make all of the choices. Instead, we will take a search and learning-based approach.
-We first define a collection of actions we can take to transform a program. Example actions include loop transformations, inlining,
-vectorization. We call these actions <strong>scheduling primitives</strong>. The collection of scheduling primitives defines a search space of possible
-optimizations we can make to a program. The system then searches over different possible scheduling
-sequence to pick the best scheduling combination.
-The search procedure is usually guided by a machine learning algorithm.</p>
-<p>We can record the best schedule sequence for an (possibly-fused) operator once the search is completed. The compiler can then just lookup the best
-schedule sequence and apply it to the program. Notably, this schedule application phase is <strong>exactly like</strong> the rule-based transformations,
-enabling us to share the same interface convention with tradition passes.</p>
-<p>We use search based optimizations to handle the initial tir function generation problem. This part of the module is called AutoTVM(auto_scheduler).
-We expect to expand the learning-based transformations to more areas as we continue to develop the TVM stack.</p>
+<li><p><strong>TensorIR schedule</strong>: TensorIR schedules are designed to optimize the TensorIR functions for a specific target, with user-guided instructions and control how the target code is generated.
+For CPU targets, TIR PrimFunc can generate valid code and execute on the target device without schedule but with very-low performance. However, for GPU targets, the schedule is essential
+for generating valid code with thread bindings. For more details, please refer to the <a class="reference internal" href="../deep_dive/tensor_ir/tutorials/tir_transformation.html#tir-transform"><span class="std std-ref">TensorIR Transformation</span></a> section. Additionally, we provides <code class="docutils literal notranslate"><span class="pre">MetaSchedule</span></code> to
+automate the search of TensorIR schedule.</p></li>
+<li><p><strong>Lowering Passes</strong>: These passes usually perform after the schedule is applied, transforming a TIR PrimFunc into another functionally equivalent PrimFunc, but closer to the
+target-specific representation. For example, there are passes to flatten multi-dimensional access to one-dimensional pointer access, to expand the intrinsics into target-specific ones,
+and to decorate the function entry to meet the runtime calling convention.</p></li>
+</ul>
+<dl class="simple">
+<dt>Many low-level optimizations can be handled in the target phase by the LLVM, CUDA C, and other target compilers. As a result, we leave low-level optimizations such as register allocation</dt><dd><p>to the downstream compilers and only focus on optimizations that are not covered by them.</p>
+</dd>
+</dl>
+</div>
+<div class="section" id="cross-level-transformations">
+<h4>cross-level transformations<a class="headerlink" href="#cross-level-transformations" title="Permalink to this headline">¶</a></h4>
+<p>Apache TVM brings a unity strategy to optimize the end-to-end models. As the IRModule includes both relax and tir functions, the cross-level transformations are designed to mutate
+the IRModule by applying different transformations to these two types of functions.</p>
+<p>For example, <code class="docutils literal notranslate"><span class="pre">relax.LegalizeOps</span></code> pass mutates the IRModule by lowering relax operators, add corresponding TIR PrimFunc into the IRModule, and replace the relax operators
+with calls to the lowered TIR PrimFunc. Another example is operator fusion pipeline in relax (including <code class="docutils literal notranslate"><span class="pre">relax.FuseOps</span></code> and <code class="docutils literal notranslate"><span class="pre">relax.FuseTIR</span></code>), which fuse multiple consecutive tensor operations
+into one. Different from the previous implementations, relax fusion pipeline analyzes the pattern of TIR functions and detects the best fusion rules automatically rather
+than human-defined operator fusion patterns.</p>
+</div>
</div>
<div class="section" id="target-translation">
<h3>Target Translation<a class="headerlink" href="#target-translation" title="Permalink to this headline">¶</a></h3>
@@ -596,15 +573,6 @@ except that the data structure of interest changes from the numpy.ndarray to tvm
</ul>
</div>
</div>
-<div class="section" id="logical-architecture-components">
-<h2>Logical Architecture Components<a class="headerlink" href="#logical-architecture-components" title="Permalink to this headline">¶</a></h2>
-<div class="figure align-center" id="id1">
-<a class="reference internal image-reference" href="https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_static_overview.svg"><img alt="https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_static_overview.svg" src="https://raw.githubusercontent.com/tlc-pack/web-data/main/images/design/tvm_static_overview.svg" width="85%" /></a>
-<p class="caption"><span class="caption-text">TVM Architecture Diagram</span><a class="headerlink" href="#id1" title="Permalink to this image">¶</a></p>
-</div>
-<p>The above figure shows the major logical components in the project. Please read the following sections
-for information about the components and their relations.</p>
-</div>
<div class="section" id="tvm-support">
<h2>tvm/support<a class="headerlink" href="#tvm-support" title="Permalink to this headline">¶</a></h2>
<p>The support module contains the most common utilities for the infrastructure, such as generic arena allocator, socket, and logging.</p>
@@ -633,7 +601,6 @@ for learning-based optimizations.</p>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="debugger.html">Debugger</a></li>
-<li class="toctree-l1"><a class="reference internal" href="virtual_machine.html">Putting the VM in TVM: The Relay Virtual Machine</a></li>
<li class="toctree-l1"><a class="reference internal" href="introduction_to_module_serialization.html">Introduction to Module Serialization</a></li>
<li class="toctree-l1"><a class="reference internal" href="device_target_interactions.html">Device/Target Interactions</a></li>
</ul>
@@ -696,10 +663,23 @@ in the target and builtin information registered to each target id(cuda, opencl)
</ul>
</div>
</div>
+<div class="section" id="tvm-relax">
+<h2>tvm/relax<a class="headerlink" href="#tvm-relax" title="Permalink to this headline">¶</a></h2>
+<p>Relax is the high-level IR used to represent the computational graph of a model. Various optimizations are defined in <code class="docutils literal notranslate"><span class="pre">relax.transform</span></code>.
+Note that Relax usually works closely the the TensorIR IRModule, most of the transformations are applied on the both Relax and TensorIR functions
+in the IRModule. Please refer to the <a class="reference internal" href="../deep_dive/relax/index.html#relax-deep-dive"><span class="std std-ref">Relax Deep Dive</span></a> for more details.</p>
+</div>
<div class="section" id="tvm-tir">
<h2>tvm/tir<a class="headerlink" href="#tvm-tir" title="Permalink to this headline">¶</a></h2>
<p>TIR contains the definition of the low-level program representations. We use <cite>tir::PrimFunc</cite> to represent functions that can be transformed by TIR passes.
-Besides the IR data structures, the tir module also defines a set of builtin intrinsics and their attributes via the common Op registry, as well as transformation passes in <cite>tir/transform</cite>.</p>
+Besides the IR data structures, the tir module also includes:</p>
+<ul class="simple">
+<li><p>A set of schedule primitives to control the generated code in <code class="docutils literal notranslate"><span class="pre">tir/schedule</span></code>.</p></li>
+<li><p>A set of builtin intrinsics in <code class="docutils literal notranslate"><span class="pre">tir/tensor_intrin</span></code>.</p></li>
+<li><p>A set of analysis passes to analyze the TIR functions in <code class="docutils literal notranslate"><span class="pre">tir/analysis</span></code>.</p></li>
+<li><p>A set of transformation passes to lower or optimize the TIR functions in <code class="docutils literal notranslate"><span class="pre">tir/transform</span></code>.</p></li>
+</ul>
+<p>Please refer to the <a class="reference internal" href="../deep_dive/tensor_ir/index.html#tensor-ir-deep-dive"><span class="std std-ref">TensorIR Deep Dive</span></a> for more details.</p>
</div>
<div class="section" id="tvm-arith">
<h2>tvm/arith<a class="headerlink" href="#tvm-arith" title="Permalink to this headline">¶</a></h2>
@@ -707,74 +687,28 @@ Besides the IR data structures, the tir module also defines a set of builtin int
arithmetic properties — the positiveness, variable bound, and the integer set that describes the iterator space. arith module provides
a collection of tools that do (primarily integer) analysis. A TIR pass can use these analyses to simplify and optimize the code.</p>
</div>
-<div class="section" id="tvm-te">
-<h2>tvm/te<a class="headerlink" href="#tvm-te" title="Permalink to this headline">¶</a></h2>
-<p>The name te stands for “tensor expression”. This is a domain-specific language module that allows us to construct <cite>tir::PrimFunc</cite> variants quickly by writing tensor expressions.
-Importantly, a tensor expression itself is not a self-contained function that can be stored into IRModule. Instead, it is a fragment of IR that we can stitch together to build an IRModule.</p>
-<p><cite>te/schedule</cite> provides a collection of scheduling primitives to control the function being generated. In the future, we might bring some of
-these scheduling components to the a <cite>tir::PrimFunc</cite> itself.</p>
-<div class="toctree-wrapper compound">
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="inferbound.html">InferBound Pass</a></li>
-<li class="toctree-l1"><a class="reference internal" href="hybrid_script.html">Hybrid Frontend Developer Guide</a></li>
-</ul>
-</div>
-</div>
-<div class="section" id="tvm-topi">
-<h2>tvm/topi<a class="headerlink" href="#tvm-topi" title="Permalink to this headline">¶</a></h2>
+<div class="section" id="tvm-te-and-tvm-topi">
+<h2>tvm/te and tvm/topi<a class="headerlink" href="#tvm-te-and-tvm-topi" title="Permalink to this headline">¶</a></h2>
+<p>TE stands for Tensor Expression. TE is a domain-specific language (DSL) for describing tensor computations. Importantly, a tensor expression
+itself is not a self-contained function that can be stored into IRModule. We can use <code class="docutils literal notranslate"><span class="pre">te.create_prim_func</span></code> to convert a tensor expression to a <code class="docutils literal notranslate"><span class="pre">tir::PrimFunc</span></code>
+and then integrate it into the IRModule.</p>
<p>While possible to construct operators directly via TIR or tensor expressions (TE) for each use case it is tedious to do so.
-<cite>topi</cite> (Tensor operator inventory) provides a set of pre-defined operators (in TE or TIR) defined by
-numpy and found in common deep learning workloads. We also provide a collection of common schedule templates to obtain performant implementations across different target platforms.</p>
-</div>
-<div class="section" id="tvm-relay">
-<h2>tvm/relay<a class="headerlink" href="#tvm-relay" title="Permalink to this headline">¶</a></h2>
-<p>Relay is the high-level functional IR used to represent full models. Various optimizations are defined in <cite>relay.transform</cite>. The Relay compiler defines multiple dialects,
-and each dialect is designed to support specific styles of optimization. Notable ones include QNN(for importing pre-quantized models), VM(for lowering to dynamic virtual machine),
-memory(for memory optimization).</p>
-<div class="toctree-wrapper compound">
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="relay_intro.html">Introduction to Relay IR</a></li>
-<li class="toctree-l1"><a class="reference internal" href="relay_op_strategy.html">Relay Operator Strategy</a></li>
-<li class="toctree-l1"><a class="reference internal" href="convert_layout.html">Convert Layout Pass</a></li>
-</ul>
+<cite>topi</cite> (Tensor operator inventory) provides a set of pre-defined operators defined by numpy and found in common deep learning workloads.</p>
</div>
+<div class="section" id="tvm-meta-schedule">
+<h2>tvm/meta_schedule<a class="headerlink" href="#tvm-meta-schedule" title="Permalink to this headline">¶</a></h2>
+<p>MetaSchedule is a system for automated search-based program optimization. It is designed to be a drop-in replacement for AutoTVM and AutoScheduler,
+and can be used to optimize TensorIR schedules. Note that MetaSchedule only works with static-shape workloads.</p>
</div>
-<div class="section" id="tvm-autotvm">
-<h2>tvm/autotvm<a class="headerlink" href="#tvm-autotvm" title="Permalink to this headline">¶</a></h2>
-<p>AutoTVM and AutoScheduler are both components which automate search based program optimization. This is rapidly evolving and primarily consists of:</p>
+<div class="section" id="tvm-dlight">
+<h2>tvm/dlight<a class="headerlink" href="#tvm-dlight" title="Permalink to this headline">¶</a></h2>
+<p>DLight is a set of pre-defined, easy-to-use, and performant TIR schedules. DLight aims:</p>
<ul class="simple">
-<li><p>Cost models and feature extraction.</p></li>
-<li><p>A record format for storing program benchmark results for cost model construction.</p></li>
-<li><p>A set of search policies over program transformations.</p></li>
+<li><p>Fully support <strong>dynamic shape workloads</strong>.</p></li>
+<li><p><strong>Light weight</strong>. DLight schedules provides tuning-free or (very few-shots tuning) schedule with reasonable performance.</p></li>
+<li><p><strong>Robust</strong>. DLight schedules are designed to be robust and general-purpose for a single rule. And if the rule is not applicable,
+DLight not raise any error and switch to the next rule automatically.</p></li>
</ul>
-<p>Automated program optimization is still an active research field. As a result, we have attempted to modularize the design so that researchers may quickly modify a
-component or apply their own algorithms via the Python bindings, and
-customize the search and plugin their algorithms from the Python binding.</p>
-<div class="toctree-wrapper compound">
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="benchmark.html">Benchmark Performance Log Format</a></li>
-</ul>
-</div>
-</div>
-<div class="section" id="frontends">
-<h2>Frontends<a class="headerlink" href="#frontends" title="Permalink to this headline">¶</a></h2>
-<p>Frontends ingest models from different frameworks into the TVM stack.
-<a class="reference internal" href="../reference/api/python/relay/frontend.html#module-tvm.relay.frontend" title="tvm.relay.frontend"><code class="xref py py-mod docutils literal notranslate"><span class="pre">tvm.relay.frontend</span></code></a> is the namespace for model ingestion APIs.</p>
-<div class="toctree-wrapper compound">
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="frontend/tensorflow.html">TensorFlow Frontend</a></li>
-</ul>
-</div>
-</div>
-<div class="section" id="microtvm">
-<h2>microTVM<a class="headerlink" href="#microtvm" title="Permalink to this headline">¶</a></h2>
-<div class="toctree-wrapper compound">
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="microtvm_design.html">microTVM Design Document</a></li>
-<li class="toctree-l1"><a class="reference internal" href="microtvm_project_api.html">microTVM Project API</a></li>
-<li class="toctree-l1"><a class="reference internal" href="model_library_format.html">Model Library Format</a></li>
-</ul>
-</div>
</div>
</div>
@@ -791,7 +725,7 @@ customize the search and plugin their algorithms from the Python binding.</p>
<a href="runtime.html" class="btn btn-neutral float-right" title="TVM Runtime System" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
- <a href="../reference/langref/hybrid_script.html" class="btn btn-neutral float-left" title="Hybrid Frontend Language Reference" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
+ <a href="../errors.html" class="btn btn-neutral float-left" title="Handle TVM Errors" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
</div>
diff --git a/docs/arch/inferbound.html b/docs/arch/inferbound.html
deleted file mode 100644
index 6a70a94698..0000000000
--- a/docs/arch/inferbound.html
+++ /dev/null
@@ -1,1080 +0,0 @@
-
-
-
-
-
-
-<!DOCTYPE html>
-<html class="writer-html5" lang="en" >
-<head>
- <meta charset="utf-8">
-
- <meta name="viewport" content="width=device-width, initial-scale=1.0">
-
- <title>InferBound Pass — tvm 0.18.dev0 documentation</title>
-
-
-
- <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
- <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
- <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
- <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-binder.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-dataframe.css" type="text/css" />
- <link rel="stylesheet" href="../_static/sg_gallery-rendered-html.css" type="text/css" />
- <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
- <link rel="stylesheet" href="../_static/css/tlcpack_theme.css" type="text/css" />
-
-
-
- <link rel="shortcut icon" href="../_static/tvm-logo-square.png"/>
-
-
-
-
-
-
-
- <script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
- <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
- <script src="../_static/jquery.js"></script>
- <script src="../_static/underscore.js"></script>
- <script src="../_static/doctools.js"></script>
-
- <script type="text/javascript" src="../_static/js/theme.js"></script>
-
-
- <script type="text/javascript" src="../_static/js/tlcpack_theme.js"></script>
- <link rel="index" title="Index" href="../genindex.html" />
- <link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Hybrid Frontend Developer Guide" href="hybrid_script.html" />
- <link rel="prev" title="Pass Infrastructure" href="pass_infra.html" />
-</head>
-
-<body class="wy-body-for-nav">
-
-
- <div class="wy-grid-for-nav">
-
-
-<header class="header">
- <div class="innercontainer">
- <div class="headerInner d-flex justify-content-between align-items-center">
- <div class="headerLogo">
- <a href="https://tvm.apache.org/"><img src=https://tvm.apache.org/assets/images/logo.svg alt="logo"></a>
- </div>
-
- <div id="headMenu" class="headerNav">
- <button type="button" id="closeHeadMenu" class="navCloseBtn"><img src="../_static/img/close-icon.svg" alt="Close"></button>
- <ul class="nav">
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/community>Community</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/download>Download</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/blog>Blog</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvm.apache.org/docs>Docs</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://tvmconf.org>Conference</a>
- </li>
- <li class="nav-item">
- <a class="nav-link" href=https://github.com/apache/tvm/>Github</a>
- </li>
- </ul>
- <div class="responsivetlcdropdown">
- <button type="button" class="btn-link">
- ASF
- </button>
- <ul>
- <li>
- <a href=https://apache.org/>Apache Homepage</a>
- </li>
- <li>
- <a href=https://www.apache.org/licenses/>License</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/sponsorship.html>Sponsorship</a>
- </li>
- <li>
- <a href=https://tvm.apache.org/docs/reference/security.html>Security</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/thanks.html>Thanks</a>
- </li>
- <li>
- <a href=https://www.apache.org/events/current-event>Events</a>
- </li>
- </ul>
- </div>
- </div>
- <div class="responsiveMenuIcon">
- <button type="button" id="menuBtn" class="btn-menu"><img src="../_static/img/menu-icon.svg" alt="Menu Icon"></button>
- </div>
-
- <div class="tlcDropdown">
- <div class="dropdown">
- <button type="button" class="btn-link dropdown-toggle" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
- ASF
- </button>
- <div class="dropdown-menu dropdown-menu-right">
- <ul>
- <li>
- <a href=https://apache.org/>Apache Homepage</a>
- </li>
- <li>
- <a href=https://www.apache.org/licenses/>License</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/sponsorship.html>Sponsorship</a>
- </li>
- <li>
- <a href=https://tvm.apache.org/docs/reference/security.html>Security</a>
- </li>
- <li>
- <a href=https://www.apache.org/foundation/thanks.html>Thanks</a>
- </li>
- <li>
- <a href=https://www.apache.org/events/current-event>Events</a>
- </li>
- </ul>
- </div>
- </div>
- </div>
- </div>
- </div>
- </header>
-
- <nav data-toggle="wy-nav-shift" class="wy-nav-side fixed">
- <div class="wy-side-scroll">
- <div class="wy-side-nav-search" >
-
-
-
- <a href="../index.html">
-
-
-
-
- <img src="../_static/tvm-logo-small.png" class="logo" alt="Logo"/>
-
- </a>
-
-
-
-
- <input type="checkbox" class="version-toggle-box" hidden id="version-toggle">
- <label for="version-toggle" class="version-toggle-label">
- <div tabindex="0" class="version version-selector version-selector-show">
- 0.18.dev0 <span class="chevron versions-hidden"><svg fill="none" height="24" viewBox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg"><path d="m8 4 8 8-8 8" stroke="#000" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"/></svg></span><span class="chevron versions-shown"><svg fill="none" height="24" viewBox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg"><path d="m4 8 8 8 8-8" stroke="#000" stroke-linecap="round" stroke-linejoin="round [...]
- </div>
- </label>
- <div class="version-details wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
- <p class="caption" role="heading"><span class="caption-text">Versions</span></p>
- <ol style="text-align: left">
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="/">0.18.dev0 (main)</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.8.0/">v0.8.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.9.0/">v0.9.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.10.0/">v0.10.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.11.0/">v0.11.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.12.0/">v0.12.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.13.0/">v0.13.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.14.0/">v0.14.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.15.0/">v0.15.0</a></div></li>
-
-
-
-
- <li><div class="version"><a style="font-size: 0.8em; padding: 4px" href="v0.16.0/">v0.16.0</a></div></li>
-
- </ol>
- </div>
-
-
-
-
-<div role="search">
- <form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
- <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
- </form>
-</div>
-
-
- </div>
-
-
- <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
-
-
-
-
-
-
- <p class="caption" role="heading"><span class="caption-text">Getting Started</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/overview.html">Overview</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../install/index.html">Installing TVM</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/tutorials/quick_start.html">Quick Start</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../get_started/tutorials/ir_module.html">IRModule</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">How To</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/e2e_opt_model.html">End-to-End Optimize Model</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/customize_opt.html">Customize Optimization</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/optimize_llm.html">Optimize Large Language Model</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/tutorials/cross_compilation_and_rpc.html">Cross Compilation and RPC</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/dev/index.html">Development Guides</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Deep Dive</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/tensor_ir/index.html">TensorIR</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../deep_dive/relax/index.html">Relax</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/python/index.html">Python API</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/api/links.html">Other APIs</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Legacy</span></p>
-<ul class="current">
-<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">User Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../how_to/legacy_index.html">How To Guides</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/tutorial/index.html">Developer Tutorial</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../dev/how_to/how_to.html">Developer How-To Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/langref/index.html">Language Reference</a></li>
-<li class="toctree-l1 current"><a class="reference internal" href="index.html">Design and Architecture</a><ul class="current">
-<li class="toctree-l2"><a class="reference internal" href="index.html#example-compilation-flow">Example Compilation Flow</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#logical-architecture-components">Logical Architecture Components</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-support">tvm/support</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-runtime">tvm/runtime</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-node">tvm/node</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-ir">tvm/ir</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-target">tvm/target</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-tir">tvm/tir</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-arith">tvm/arith</a></li>
-<li class="toctree-l2 current"><a class="reference internal" href="index.html#tvm-te">tvm/te</a><ul class="current">
-<li class="toctree-l3 current"><a class="current reference internal" href="#">InferBound Pass</a><ul>
-<li class="toctree-l4"><a class="reference internal" href="#itervar-hyper-graph">IterVar Hyper-graph</a></li>
-<li class="toctree-l4"><a class="reference internal" href="#passdowndomain">PassDownDomain</a></li>
-<li class="toctree-l4"><a class="reference internal" href="#inferrootbound">InferRootBound</a></li>
-<li class="toctree-l4"><a class="reference internal" href="#inferbound-with-compute-at">InferBound with compute_at</a></li>
-</ul>
-</li>
-<li class="toctree-l3"><a class="reference internal" href="hybrid_script.html">Hybrid Frontend Developer Guide</a></li>
-</ul>
-</li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-topi">tvm/topi</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-relay">tvm/relay</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#tvm-autotvm">tvm/autotvm</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#frontends">Frontends</a></li>
-<li class="toctree-l2"><a class="reference internal" href="index.html#microtvm">microTVM</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="../topic/microtvm/index.html">microTVM: TVM on bare-metal</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../topic/vta/index.html">VTA: Versatile Tensor Accelerator</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">About</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../contribute/index.html">Contributor Guide</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/publications.html">Publications</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../reference/security.html">Security Guide</a></li>
-</ul>
-<p class="caption" role="heading"><span class="caption-text">Index</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="../genindex.html">Index</a></li>
-</ul>
-
-
-
- </div>
-
- </div>
- </nav>
-
- <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
-
- <nav class="wy-nav-top" aria-label="top navigation" data-toggle="wy-nav-top">
-
- <div class="togglemenu">
-
- </div>
- <div class="nav-content">
- <!-- tvm -->
- Table of Contents
- </div>
-
- </nav>
-
-
- <div class="wy-nav-content">
-
- <div class="rst-content">
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-<div role="navigation" aria-label="breadcrumbs navigation">
-
- <ul class="wy-breadcrumbs">
-
- <li><a href="../index.html">Docs</a> <span class="br-arrow">></span></li>
-
- <li><a href="index.html">Design and Architecture</a> <span class="br-arrow">></span></li>
-
- <li>InferBound Pass</li>
-
-
-
-
-
-
-
-
- <li class="wy-breadcrumbs-aside">
-
-
-
- <a href="https://github.com/apache/tvm/edit/main/docs/arch/inferbound.rst" class="fa fa-github"> Edit on GitHub</a>
-
-
-
- </li>
-
- </ul>
-
-
- <hr/>
-</div>
- <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
- <div itemprop="articleBody">
-
- <div class="section" id="inferbound-pass">
-<span id="dev-inferbound-pass"></span><h1>InferBound Pass<a class="headerlink" href="#inferbound-pass" title="Permalink to this headline">¶</a></h1>
-<p>The InferBound pass is run after normalize, and before ScheduleOps <a class="reference external" href="https://github.com/apache/tvm/blob/main/python/tvm/driver/build_module.py">build_module.py</a>. The main job of InferBound is to create the bounds map, which specifies a Range for each IterVar in the program. These bounds are then passed to ScheduleOps, where they are used to set the extents of For loops, see <a class="reference external" href="https://github.com/apache/tvm/blob/main [...]
-<p>The output of InferBound is a map from IterVar to Range:</p>
-<div class="highlight-cpp notranslate"><div class="highlight"><pre><span></span><span class="n">Map</span><span class="o"><</span><span class="n">IterVar</span><span class="p">,</span><span class="w"> </span><span class="n">Range</span><span class="o">></span><span class="w"> </span><span class="n">InferBound</span><span class="p">(</span><span class="k">const</span><span class="w"> </span><span class="n">Schedule</span><span class="o">&</span><span class="w"> </span><span clas [...]
-</pre></div>
-</div>
-<p>Therefore, let’s review the Range and IterVar classes:</p>
-<div class="highlight-cpp notranslate"><div class="highlight"><pre><span></span><span class="k">namespace</span><span class="w"> </span><span class="nn">HalideIR</span><span class="w"> </span><span class="p">{</span>
-<span class="k">namespace</span><span class="w"> </span><span class="nn">IR</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="k">class</span><span class="w"> </span><span class="nc">RangeNode</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="k">public</span><span class="w"> </span><span class="n">Node</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="k">public</span><span class="o">:</span>
-<span class="w"> </span><span class="n">Expr</span><span class="w"> </span><span class="n">min</span><span class="p">;</span>
-<span class="w"> </span><span class="n">Expr</span><span class="w"> </span><span class="n">extent</span><span class="p">;</span>
-<span class="w"> </span><span class="c1">// remainder omitted</span>
-<span class="w"> </span><span class="p">};</span>
-<span class="w"> </span><span class="p">}}</span>
-
-<span class="k">namespace</span><span class="w"> </span><span class="nn">tvm</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="k">class</span><span class="w"> </span><span class="nc">IterVarNode</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="k">public</span><span class="w"> </span><span class="n">Node</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="k">public</span><span class="o">:</span>
-<span class="w"> </span><span class="n">Range</span><span class="w"> </span><span class="n">dom</span><span class="p">;</span>
-<span class="w"> </span><span class="n">Var</span><span class="w"> </span><span class="n">var</span><span class="p">;</span>
-<span class="w"> </span><span class="c1">// remainder omitted</span>
-<span class="w"> </span><span class="p">};</span>
-<span class="p">}</span>
-</pre></div>
-</div>
-<p>Note that IterVarNode also contains a Range <code class="docutils literal notranslate"><span class="pre">dom</span></code>. This <code class="docutils literal notranslate"><span class="pre">dom</span></code> may or may not have a meaningful value, depending on when the IterVar was created. For example, when <code class="docutils literal notranslate"><span class="pre">tvm.compute</span></code> is called, an <a class="reference external" href="https://github.com/apache/tvm/blob/main/src [...]
-<p>On the other hand, when <code class="docutils literal notranslate"><span class="pre">tvm.split</span></code> is called, <a class="reference external" href="https://github.com/apache/tvm/blob/main/src/te/schedule/schedule_lang.cc">IterVars are created</a> for the inner and outer axes, but these IterVars are not given a meaningful <code class="docutils literal notranslate"><span class="pre">dom</span></code> value.</p>
-<p>In any case, the <code class="docutils literal notranslate"><span class="pre">dom</span></code> member of an IterVar is never modified during InferBound. However, keep in mind that the <code class="docutils literal notranslate"><span class="pre">dom</span></code> member of an IterVar is sometimes used as default value for the Ranges InferBound computes.</p>
-<p>We next review some TVM codebase concepts that are required to understand the InferBound pass.</p>
-<p>Recall that InferBound takes one argument, a Schedule. This schedule object, and its members, contains all information about the program being compiled.</p>
-<p>A TVM schedule is composed of Stages. Each stage has exactly one Operation, e.g., a ComputeOp or a TensorComputeOp. Each operation has a list of root_iter_vars, which in the case of ComputeOp, are composed of the axis IterVars and the reduce axis IterVars. Each operation can also contain many other IterVars, but all of them are related by the operations’s list of IterVarRelations. Each IterVarRelation represents either a split, fuse or rebase in the schedule. For example, in the case [...]
-<div class="highlight-cpp notranslate"><div class="highlight"><pre><span></span><span class="k">namespace</span><span class="w"> </span><span class="nn">tvm</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="k">class</span><span class="w"> </span><span class="nc">ScheduleNode</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="k">public</span><span class="w"> </span><span class="n">Node</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="k">public</span><span class="o">:</span>
-<span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Operation</span><span class="o">></span><span class="w"> </span><span class="n">outputs</span><span class="p">;</span>
-<span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Stage</span><span class="o">></span><span class="w"> </span><span class="n">stages</span><span class="p">;</span>
-<span class="w"> </span><span class="n">Map</span><span class="o"><</span><span class="n">Operation</span><span class="p">,</span><span class="w"> </span><span class="n">Stage</span><span class="o">></span><span class="w"> </span><span class="n">stage_map</span><span class="p">;</span>
-<span class="w"> </span><span class="c1">// remainder omitted</span>
-<span class="w"> </span><span class="p">};</span>
-
-<span class="w"> </span><span class="k">class</span><span class="w"> </span><span class="nc">StageNode</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="k">public</span><span class="w"> </span><span class="n">Node</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="k">public</span><span class="o">:</span>
-<span class="w"> </span><span class="n">Operation</span><span class="w"> </span><span class="n">op</span><span class="p">;</span>
-<span class="w"> </span><span class="n">Operation</span><span class="w"> </span><span class="n">origin_op</span><span class="p">;</span>
-<span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">IterVar</span><span class="o">></span><span class="w"> </span><span class="n">all_iter_vars</span><span class="p">;</span>
-<span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">IterVar</span><span class="o">></span><span class="w"> </span><span class="n">leaf_iter_vars</span><span class="p">;</span>
-<span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">IterVarRelation</span><span class="o">></span><span class="w"> </span><span class="n">relations</span><span class="p">;</span>
-<span class="w"> </span><span class="c1">// remainder omitted</span>
-<span class="w"> </span><span class="p">};</span>
-
-<span class="w"> </span><span class="k">class</span><span class="w"> </span><span class="nc">OperationNode</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="k">public</span><span class="w"> </span><span class="n">Node</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="k">public</span><span class="o">:</span>
-<span class="w"> </span><span class="k">virtual</span><span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">IterVar</span><span class="o">></span><span class="w"> </span><span class="n">root_iter_vars</span><span class="p">();</span>
-<span class="w"> </span><span class="k">virtual</span><span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Tensor</span><span class="o">></span><span class="w"> </span><span class="n">InputTensors</span><span class="p">();</span>
-<span class="w"> </span><span class="c1">// remainder omitted</span>
-<span class="w"> </span><span class="p">};</span>
-
-<span class="w"> </span><span class="k">class</span><span class="w"> </span><span class="nc">ComputeOpNode</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="k">public</span><span class="w"> </span><span class="n">OperationNode</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="k">public</span><span class="o">:</span>
-<span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">IterVar</span><span class="o">></span><span class="w"> </span><span class="n">axis</span><span class="p">;</span>
-<span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">IterVar</span><span class="o">></span><span class="w"> </span><span class="n">reduce_axis</span><span class="p">;</span>
-<span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">Expr</span><span class="o">></span><span class="w"> </span><span class="n">body</span><span class="p">;</span>
-<span class="w"> </span><span class="n">Array</span><span class="o"><</span><span class="n">IterVar</span><span class="o">></span><span class="w"> </span><span class="n">root_iter_vars</span><span class="p">();</span>
-<span class="w"> </span><span class="c1">// remainder omitted</span>
-<span class="w"> </span><span class="p">};</span>
-<span class="p">}</span>
-</pre></div>
-</div>
-<p>Tensors haven’t been mentioned yet, but in the context of TVM, a Tensor represents output of an operation.</p>
-<div class="highlight-cpp notranslate"><div class="highlight"><pre><span></span><span class="k">class</span><span class="w"> </span><span class="nc">TensorNode</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="k">public</span><span class="w"> </span><span class="n">Node</span><span class="w"> </span><span class="p">{</span>
-<span class="k">public</span><span class="o">:</span>
-<span class="w"> </span><span class="c1">// The source operation, can be None</span>
-<span class="w"> </span><span class="c1">// This Tensor is output by this op</span>
-<span class="w"> </span><span class="n">Operation</span><span class="w"> </span><span class="n">op</span><span class="p">;</span>
-<span class="w"> </span><span class="c1">// The output index from the source operation</span>
... 14581 lines suppressed ...